Informational Entropy of B-ary Trees after a Vertex Cut

Size: px

Start display at page:

Download "Informational Entropy of B-ary Trees after a Vertex Cut"

Nigel Wheeler
5 years ago
Views:

1 Entropy 008, 0, ; DOI: /e Article OPEN ACCESS entropy ISSN Informational Entropy of B-ary Trees after a Vertex Cut Lorentz Jäntschi,, * and Sorana D. Bolboacă, Technical University of Cluj-Napoca, Muncii Bvd, Cluj-Napoca, Cluj, Romania. Phone: Fax: lori@academicdirect.org "Iuliu Haţieganu" University of Medicine and Pharmacy Cluj-Napoca, Department of Medical Informatics and Biostatistics, 6 Louis Pasteur, Cluj-Napoca, Cluj, Romania. Phone: Fax: sbolboaca@umfcluj.ro * Author to whom correspondence should be addressed. Received: 4 August 008; in revised form: 5 October 008 / Accepted: 7 October 008 / Published: 3 November 008 Abstract: Together with stars and paths, b-ary trees are one of the most studied acyclic graph structures. As any other structure, a b-ary tree can be seen as containing information. The aim of the present research was to assess through informational entropy the structural information changes in b-ary trees after removal of an arbitrary vertex. Keywords: B-ary tree, informational entropy, vertex cut. AMS Mathematics Subject Classification: 05C05 (Trees), 37B40 (Topological entropy), 94A7 (Measures of information, entropy), T06 (Polynomials).. Introduction B-ary trees are, together with stars and paths, one of the most studied acyclic graph structures. A survey on the available literature showed that almost 0% of papers dealing with graphs deal also with trees. The statistics calculated on trees are used in machine learning [] and solving of nondeterministic polynomial-time hard (NP-hard) problems []. The concept of entropy in conjunction with regular trees was used as measure of similarity between two grammars [3]. Moreover, the information flows in b-ary trees was subject of some theoretical results, including a converse in [4]. General formulas characterizing the random walk in a b-ary tree were obtained in [5]. A general

2 Entropy 008, formula giving the number of substructures by their sizes by systematically removal of a vertex in a b- ary tree was also obtained [6]. As any other structure, a tree can be seen as a system that contains information. The aim of the present research was to assess through informational entropy the structural information changes in b- ary trees after removal of an arbitrary vertex.. Material Let T(b,) be a regular b-ary tree with levels (Figure ). The removal of an arbitrary vertex applied on a b-ary tree will generate one (when the removal is applied to a leaf), two (when the removal is applied to a root), or three (when the removal is applied to an inside node) sub-graphs. Figure. A b-ary tree with levels, T(b,) b b b 0 It is possible to evaluate the number of sub-graphs by removing every vertex from the tree one at a time. A statistical motivation is associated to this procedure. If we are able to obtain the entire population of sub-structures by their sizes after this procedure, then the probability of appearances of a substructure of a given size can be obtained (by dividing its occurrence to the total number of substructures). It could be said that systematically removal of every vertex is not a usual procedure for a given practical problem. Indeed, but usually in a real application, an issue may be the removal of one vertex when the specification of which one vertex is to be remove is unknown. One example may be a tree network topology (let us say a Regional Internet Registry providing Internet resource allocations). In this case, the assessment of the impact of removing a node is an important issue for projecting the network. The assumptions applied in the present research were as follows: all vertices and edges in the b-ary tree are present (or available) at the initial moment (before removing the first vertex), the tree is balanced, and the vertices and edges have no precedence/weight one to another. Then we will move from complete characterization of a set (the set of all sub-graphs obtained by removing of an arbitrary vertex of a tree) to an issue of probability (obtaining the appearance probability of a set of a given size). In terms of the entire structure (the tree), the appearance probability of a set of a given size may not be a relevant information but a global parameter which to characterize the tree under the given circumstances (removal of one vertex) could be of interest and was investigated.

3 Entropy 008, Method Entropy is the parameter that characterizes the amount of disorder. Rényi [5] gave a generalization of the entropy through a family of functional for quantifying the diversity, uncertainty or randomness of a system. Let X be a discrete variable that take the values {x,, xn} and p( ) the probability mass function (gives the probability that a discrete random variable is exactly equal to a value). The Rényi entropy (abbreviated as H α (X) for any non-negative order α, where α 0) is given by: H α (X) = log p (x ) α () n α i i= Two particular cases of Eq() are taken into consideration: α (or Shannon's entropy [8]), and α = (negative logarithm of Simpson's diversity index D [0] or Onicescu's energy [9]): n H (X) = p(x )log (p(x )) i i i= n D(X) p (x i) i= = ; H(X) = logd(x) () 4. Results Let us recall the structure from Figure. The following formula (abbreviated as NSSP) describes the number of substructures (coefficient of the X variable) by sizes (power of the X variable) for systematically removal of a vertex from a b-ary tree [6]: NSSP(b, ) = k= k b X + k b b + X + + k b b b + b + X b 0 (3) The Eq(3) give the full description of the counts and sizes of substructures for any,b>0. The Table presents the substructures and sizes for some particular cases of trees. Note that the coefficient of X 0 counts the total number of vertices in the original structure. Table. Substructures obtained by remove of an arbitrary vertex: regular b-ary trees b Formula Comments X 0 +X A graph with two vertices and one edge > (+)X 0 +X(X -)/(X-) For b= tree degenerates in a path > (b+)x 0 +b(x+x b ) For = tree degenerates in a star b (b +b+)x 0 +b(x b+ +X b )+b (X+X b+b ) Powers of X's may be repeated in the series = number of levels; b = ramification degree of the b-ary tree By using the Eq(3), we want to find the roots of the following equation:

4 Entropy 008, b + k + + = b b i.e. k k k k k (+ ) + b = b + b (4) Excluding b= (b,, k, and k are natural not null numbers; and is also a natural number) the equality Eq(4) can be satisfied just when the only solutions of Eq(5) is for k =+ (that implying as consequence to have the b=). The conclusion that can be drawn from here is that for b> all terms from Eq(3) are distinct. Since the problem of occurrences of sub-graphs sizes (n o ) was solved in the general case for b> (n o repeats in terms of Eq(3)) we are able to perform the calculations for a given and b (note that the problem can be extended to any b and less than or equal to a given value): + k b + + k + + k b b b n o b + k b = coef (X b b ) = n o b b = coef (X ) (5) in addition, the probability (of observing a structure of a given size in a random cut) is given by: + k b b p = p b + b b + k k b = b k= k k b (b ) = (b ) (6) The replacement of Eq(6) into Eq() and Eq() leads to the following formulas: k b (b ) H (b,)=- log (b ) k= k + b (b ) (b ) b ( ) b + b = log log b (b ) b (b )(b ) k b (b ) b + b b H (b,)=-log =-log =-log k= 4(b ) b b + b + b b + (7) (8) where H (b,) is the Rényi entropy of order ; H (b,) is the Rényi entropy of order of a b-ary tree T(b,) after removal of a vertex, when b>. The value of is given by the fact that we have equal probabilities for fragments of size (b +-k - )/(b - ), and (b + b +-k )/(b - ) respectively (see Eq(6)). For b= (path) an entry from Table can help us to obtain the values for H (,)=H () and H (,)=H (), respectively: = = = k= k= (9) H () log log log () H () = log = log = log() = H () k= (0) 5. Discussion In terms of statistics, the random event subject of the investigation was to remove a vertex from a b- ary tree, and the observation was to count the size of a connected structure (the size of a connected structure being the random variable). This statistical experiment allowed obtaining an important informational statistics: entropy and diversity index.

5 Entropy 008, Equations (7)-(0) allow one to compute the values and based on the obtained results, to plot the dependence of the H x (b,) (where x =, ; H (b,) = Rényi entropy of order ; H (b,) = Rényi entropy of order ) and D(b,) by the number of levels () and ramification degree (b) of the tree. These dependencies ranging from to 0 for and b are presented in Figure. Figure. Rényi entropy of order - H = H (b,), Rényi entropy of order - H = H (b,), and Simpson's diversity index - D = D(b,) of a b-ary tree with levels after a vertex removal H (b,) H (b,) b D(b,) b b The analysis of the calculated values obtained by applying the formulas for Rényi entropy of order and, and of Simpson's diversity index revealed the followings: For a given value of b (ramification degree of the b-ary tree): o The values of Rényi entropy of order are equal with the values of and Rényi entropy of order for b= and 0. o The distribution of D(b,), H (b,) and H (b,) follow a similar pattern for b > and 0.

6 Entropy 008, 0 58 o o Starting with b=4 the values of D(b,), H (b,) and H (b,) become a monotone function with the increases of the number of levels. Very small differences were observed for 7 b 0. If the b= is excluded from the analysis, the values of D(b,) decrease with the increasing of while the values of H (b,) and H (b,) increase with increasing of (see Figure 3). Figure 3. Variation of Simpson's diversity index (D) and Rényi's entropy of order (H ) for a given value of ramification degree..0 Diversity index (D) (number of levels) b= b= b=3 b=4 b=5 b=6 b=7 b=8 b=9 b=0 b= b= b=3 b=4 b=5 b=6 b=7 b=8 b=9 b= Renyi's entropy of order (H ) (number of levels) b= b= b=3 b=4 b=5 b=6 b=7 b=8 b=9 b=0 b= b= b=3 b=4 b=5 b=6 b=7 b=8 b=9 b=0

7 Entropy 008, 0 58 o The maximum value is systematically obtained by b= for D(b,) and b=0 for H (b,) and H (b,). The D(b,) values tend to varied with very small quantities (the differences are not observed at the level of first four decimals) starting with a certain value of (D(b,): D(,), D(3,9), D(4,7), D(5,7), D(6,7), D(7,5), D(8,5), D(9,5), D(0,5), D(,4)). The variation with very small quantities are also observed for H (b,) (H (,9); H (3,); H (4,0); H (5,8); ) and H (b,) (H (,8); H (3,0); H (4,9); H (5,8); ). For a given value of (number of levels of the b-ary tree): o For =: the values of D(b,) were equal with 0.5 for b 0; the values of H (b,) and H (b,) were equal to for b 0. o The distribution of D(b,) follows a certain pattern for b> and 0. A similar pattern is observed for distribution of H (b,) and H (b,). o The values of D(b,) increase with increases of b while the values of H (b,) and H (b,) decrease with increasing of b after an initial increase (see b> and >8, Figure 4). Figure 4. Variation of Rényi's entropy of order (H ) and (H ) for a given value of Renyi's entropy of order (H ) b (degree of ramification) = = =3 =4 =5 =6 =7 =8 =9 =0 = = =3 =4 =5 =6 =7 =8 =9 =0 The maximum value of Rényi entropy of order was equal with the maximum value of Rényi entropy of order (4.39) for T(,0). With a few exceptions (for T(,<5)) the values of Rényi entropy were higher than which leads to the conclusion that the most disorder structure were the ones from the path (unary-tree) class (T(,5), T(,6), T(,7), T(,8), T(,9), T(,0), T(,), T(,), T(,3), T(,4), T(,5), T(,6), T(,7), T(,8), T(,9), and T(,0)). The structures become more stable as the degree of the ramification of the b-ary tree increase (structures become more complex). The values of Rényi entropy of order range from.76 to.305 for T(0, < 0), and the values of Rényi entropy of order range from.37 to.444 for T(0, < 0).

8 Entropy 008, A regression analysis for a give value of levels () was applied in order to identify the relationship between Simpson's diversity index (D(b,)) and Rényi entropy of order (H (b,)), and between Rényi entropy of order (H (b,)) and (H (b,)). The obtained results expressed as equations and associated squared correlation coefficients are presented in Table. The relative variation (min=0; max=) of regression equation coefficients from Table for H = f(d) and H = f(h) are graphically presented in Figure 5. Table. Substructures by removal of an arbitrary vertex for some regular b-ary trees H = f(d) H = f(h ) Equation R (for <b<) Equation R (for <b<) D D H -.37 H D D H H D D H H D D H H D D H H D D H H D D H H D D H H D D H H D D H H D - 9. D H H D D H H D D H H D D H H D D H H D D H H D D H H D D H H D D H H coef(d ) = e -/.39 coef(d) = e -/.4 Intercept = e -/ coef(h ) = e -/.35 coef(h ) = e -/.39 Intercept = e -/ The analysis of the results presented in Table revealed the followings: Very good relationships in terms of determination coefficient are obtained between Simpson's diversity index and Rényi entropy of order, and Rényi entropy of order and, respectively. The relationships are polynomial of second order for both investigated situations ((D(b,))- (H (b,)) and (H (b,))-(h (b,)). The H (b,) = f(h (b,)) obtained slightly better performances in terms of determination coefficients compared with H (b,) = f(d(b,)) (the maximum value of R = =4 compared with R = =3). Moreover, the distribution of the determination coefficient

9 Entropy 008, is more narrow for H (b,) = f(h (b,)) (a difference between maximum and minimum of ) compared to H (b,) = f(d(b,)) (a difference between maximum and minimum of ). Figure 5. Relative variation (min=0; max=) of regression equation coefficients from Table for H = f(d) and H = f(h ) coef_d^_h coef_d_h intercept_d_h coef_h^_h coef_h_h intercept_h_h Coefficient relative value (number of levels) The polynomial of second order dependency of H (b,) by the D(b,) revealed to be done by the same equation if the number of decimals are reduced to two for 3. The polynomial of second order dependency of H (b,) by the H (b,) revealed to be done by the same equation if the number of decimals are reduced to two for. Moreover, the determination coefficient of was systematically obtained for 6. The calculated values of information entropy proved to be negatively correlated for the arbitrary vertex removal from a b-ary tree. The variation of the correlations between these measures of information is graphically presented in Figure 6. The linear relationship between Rényi's entropy of order (H ) and (H ) of b-ary trees is perfect (equal to ) for b= due to the equality of the values of these two parameters. The analysis of the obtained values of correlation coefficients revealed the followings: Degree of ramification (b): o r(h,h ): the minimum as well as the maximum values of correlation coefficient (expressed as absolute values) increase with increasing of ramification degree. Moreover, a certain pattern of variation could be observed. o r(h,d): the correlation coefficients analysed as absolute value increase with increasing of ramification degree (the range of this variation is wider ( ) compared with the narrow variation of r(h, H ) ( )). The distribution of r(h,d) follows two patterns according with the value of b (see Figure 7).

10 Entropy 008, Figure 6. Correlation between Rényi's entropy of order (H ) and (H ), and between Rényi's entropy of order (H ) and Simpson's diversity index (D) of a b-ary tree after a vertex removal r(h,h ) -r(h,d) b b The correlations between H and H,and between H and D were calculated by using the following formula: r(h,h ) = r(b,) = r([h (bb,)] bb b;,[h (bb,)] bb b; ) where [ ] denotes a matrix having b columns and rows. Figure 7. Distribution of r(h,d) with number of levels () (number of levels) Correlation coefficient b= b= b=3 b=4 b=5 b=6 b=7 b=8 b=9 b=0 b= b= b=3 b=4 b=5 b=6 b=7 b=8 b=9 b=0 Number of levels (): o The minimum and maximum value (expressed as absolute values) increase with number of levels for r(h,h ) as well as for r(h,d). o The maximum value of r(h,h ) is obtained for T(0,0). The highest correlation coefficient

11 Entropy 008, of r(h,d) was also obtained by T( 6,0). o The absolute values of r(h,h ) follows a similar pattern for 9 with a slowly decrease until b=6 followed by an increase. For >9, the absolute value of correlation coefficients increase with increasing of b. o The variation of the r(h,d) is presented in Figure 8. Overall, the correlations coefficients revealed to be good, being always higher than (r(h,h ) or smaller than (r(h,d)). The linear relationships between H and H, as well as between H and D became weaker with the increases of the degree of ramification (b) and of the number of levels (). The highest correlation expressed as the absolute value was of r(h,d) = for T(,) and of r(h,h ) = for T(,)). Figure 8. Variation of r(h,d) with the degree of ramification (b) Degree of ramification (b) Correlation coefficient = =3 =4 =5 =6 =7 =8 =9 =0 = = =3 =4 =5 =6 =7 =8 =9 =0 The informational entropy computed on tress found its application on pattern recognition [6], clarification on self-similarity properties of river networks [7], characterization of kinetic equation [8], evaluation of comminution process [9], and so on. Moreover, the informational entropy analysis provide the information about the total loss of information of the investigated structure (the maximum entropy of a system correspond to the total loss of information), asymmetry or symmetry as well as about simplicity [0]. The formulas given by Eq(7) and Eq(8) may be used for larger b and as approximation formulas when the b-ary tree is not complete (having few gaps), by using an averaged (approximated) value for its number of levels, since few for larger trees may affect in small amount the entire information contained in the tree. Figures and 6 sustain these assumptions; the entropy of the b-ary tree varies very slowly for larger values of b and.

12 Entropy 008, As it could be observed, just the balanced b-ary trees were considered in the present research and this could be a limitation of the study due to the possibility to obtain a generalization. The behaviour obtained in this research on balanced b-ary trees will be assessing in further researches on unbalanced trees (note that the unbalanced trees are particular cases). Conclusions The values of Rényi entropy of order are equal with the values of and Rényi entropy of order for a value of the ramification degree of a b-ary tree equal to. The values obtained by Rényi entropy of order and decreases with the increases of the ramification degree of the b-ary tree. The Rényi entropy of order and increase with the increases of the number of levels for a give value of the ramification degree of a b-ary tree. The relationship between Rényi entropy of order and, and between Rényi entropy of order and Simpson's diversity index revealed to be of order two polynomial types rather than of a linear type. Acknowledgements The authors wish to thank Dr. Peter Harremoës for his efforts, kindness and patients that help us to improve the manuscript. The research was supported by national research grants (LJ: ID05/UEFISCSU and SDB: ID0458/UEFISCSU). References. Carrasco, R. C.; Oncina, J.; Calera-Rubio, J. Stochastic inference of regular tree languages. Mach. Learn. 00, 44, Hu,.; Shi, Z. A subdiffusive behaviour of recurrent random walk in random environment on a regular tree. Probab. Theory. Relat. Fields. 007, 38, Calera-Rubio, J.; Carrasco, R. C. Computing the relative entropy between regular tree languages. Inform. Process. Lett. 998, 68, Mossel, E.; Peres,. Information flow on trees. Ann. Appl. Probab. 003, 3, Aldous, D. J. Random walk covering of some special trees. J. Math. Anal. Appl. 99, 57, Jäntschi, L.; Stoenoiu, C. E.; Bolboacă, S. D. A Formula for Vertex Cuts in b-trees. Int. J. Pure App. Math. 008, 47, Estrada, E.; Hatano, N. Statistical-mechanical approach to subgraph centrality in complex networks. Chem. Phys. Lett. 007, 439, Shannon, C. E. A Mathematical Theory of Communication. Bell. Syst. Techn. J. 948, 7, , Onicescu, O. Energie informationelle. Compt. Rend. Acad. Sci. Paris Ser. 966, 63, Simpson, E. H. Measurement of Diversity. Nature 949, 63, Bolboacă, S. D.; Pică, E. M.; Cimpoiu, C. V.; Jäntschi, L. Statistical Assessment of Solvent Mixture Models used for Separation of Biological Active Compounds. Molecules 008, 8,

13 Entropy 008, Friedman, W. F. The Index of Coincidence and Its Applications in Cryptography; Riverbanks Publication: Geneva, Switzerland, Illinois, USA, 9;. 3. Gass, F. Solving a Jules Verne Cryptogram. Mathem. Mag. 986, 59, Harremoes, P.; Topsoe, F. Inequalities between entropy and index of coincidence derived from information Diagrams. IEEE Trans. Inform. Theory 00, 47, Rényi, A. On measures of information and entropy. Proceedings of the 4th Berkeley Symposium on Mathematics, Statistics and Probability, 96, Jumarie, G. Informational similarity of graphs in syntactic pattern recognition. Pattern Recogn. Lett. 994, 5, Claps, P.; Fiorentino, M.; Oliveto, G. Informational entropy of fractal river networks. J. Hydrol. 996, 87, Masiuk, S.; Rakoczy, R. Kinetic equation of grinding process in mixing of granular material using probability density functions, transient operators and informational entropy. Chem. Eng. Process 008, 47, Otwinowski, H. Energy and population balances in comminution process modelling based on the informational entropy. Powder Technol. 006, 67, Lin, S-K. Correlation of Entropy with Similarity and Symmetry. J. Chem. Inf. Comput. Sci. 996, 36, by the authors; licensee Molecular Diversity Preservation International, Basel, Switzerland. This article is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (

Sorana-Daniela BOLBOACĂ a, Lorentz JÄNTSCHI b. Abstract

Sorana-Daniela BOLBOACĂ a, Lorentz JÄNTSCHI b. Abstract Computer-Aided Chemical Engineering, ISSN 1570-7946, 24(2007) 2007 Elsevier B.V./Ltd. All rights reserved. 965 Modelling the Inhibition Activity on Carbonic Anhydrase I of Some Substituted Thiadiazole-