Information Representation by Hierarchies

Transcription

1 Information Representation by Hierarchies We wish to create a telephone system which will enable each telephone to communicate with every other one There are in total N telephones Solution? We can connect each telephone with the other one, N(N-1)/2 connections For big N, the number of connections would be to big 1

2 We could connect each telephone to a single central office For N telephones this would require only N connections The switching load of the central office would be enormous Long waiting times If the telephones are far away, we would require long wires Emphasis of the hierarchical arrangement Set the central office at the top of the diagram Hierarchical arrangement of the network with 2 levels 2

3 Suppose k central offices each serve N/k telephones directly k central offices are themselves connected to a single national office 3-level hierarchical network topology N+k connections What is the optimal number of k? Stating from a given telephone, the waiting time to obtain service from its central office will be proportional to the number of telephones, N/k Time from national office is k Total time k+n/k, minimum for k = Average total time is proportional to Star network average time is N N 2" N Would additional level bring an improvement? 3

4 Our structure, a tree is organized in levels Suppose each node belonging to a given level has the same number of children The levels are labeled from 0 to L, level 0 is the root, level L are the leafs How many nodes should each level contain in order that the average waiting time required is as small as possible? The average witing/search time through such a tree will be minimized if the number of nodes N(k) which belongs to the level k is related to the number N=N(L) N(k)=N k/l for k=1,...,l Example L=2, N(1)=N 1/2 4

5 N(k)=N k/l for k=1,...,l Whatever the value of L is, each subtree of a tree which minimizes average waiting/search time must be also minimize the average time Otherwise it would be possible to replace the subtree with a more efficient Proof of N(k)=N k/l is based on induction L=1 is true We will assume the validity of the equation for k < L and prove it for k=l 5

6 From the equation it follows (N(0)=1) N(1)=N 1/l N(1) k =N k/l =N(k) N(k)=N(1) k N(1)=N(1) 1 N(2)=N(1) 2 N(3)=N(1) 3 and so on...till N(L-1)=N(1) L-1, prove it for L 2t average = N(1)/N(0)+N(2)/N(1)+N(3)/N(2)...+N(L)/N(L-1) Factor 2 appears because, on the average only half of the nodes at each level are encountered in a sequential search 2t average = N(1)+N(1)+N(1)+...+N(L)/N(1) L-1 2t average =(L-1)*N(1)+N(L)/N(1) L-1 2t average =(L-1)*N(1)+N(L)/N(1) L-1 The total number of nodes N (=N(L)) is given, our task is to determine N(1) so that t is minimized dt/n(1)=0 0=dt/N(1)=1/2*((L-1) - (L-1)*N(L)/N(1) L ) Solution --> N=N(L)=N(1) L Which is the formula for k=l, the induction step is conducted 6

7 With N=N(L) follows t avarage =L/2*N(1)=L/2*N 1/L Value of L which minimize tavarage t avarage =L/2*N 1/L =L/2*exp(1/L*log(N)) dt/dl=1/2(1-log(n)/l)*exp(1/l*log(n))=0 L=log(N) 0.69*log 2 (N) t min =log(n)/2*exp(log(n)/(log N)) t min =log(n)/2*exp(1) t min 0.94*log 2 (N) L 0.69*log 2 (N) t min 0.94*log 2 (N) As an example consider a library catalog card tray, with N=800 card, L=7 will minimize the search time However, traditional library card catalogs seldom have an indexing hierarchy greater than 3 7

8 Hierarchical structures occur through civilization Decomposition of a hierarchy into simpler parts Space Shuttle, Cars By repetition of the process of partitioning the system into disjoint and independent sub-units, a level of simplicity is finally reached for which individual expertise suffices to design and produce these elementary components The complex combination which carry the genetic code could hardly, in the limited time, since cooling of the earth, have come into direct existence, if not by hierarchical organization DNA consists of building blocks, each of which in turn is constructed from simpler standardized chemical combinations These combinations have a higher probability to come together 8

9 How much information is carried by DNA? The information of DNA is coded in terms of ordered triples of four chemical substances called nucleotide bases Ordered triples of nucleotide bases correspond to the 20 primary amino acids which are the building blocks of proteins Because there are 64 (4 3 ) possible combinations, it follows that a given amino acid may correspond to more than one combination (=condon) Human DNA consists of 5 bilion nucleid bases, 1.7 billion condons Each condon corresponds to 6 bits, since 2 6 =64 9

10 Structure of mater itself is hierarchically organized 10

11 Elementary arithmetic's A primitive and ancient way to denote a positive whole number N is to arrange N similar strokes Counting notation, take up large amount of space Babylonian invention of more efficient positional notation for numbers (4000 years ago) Positional notation is itself a hierarchical organized structure Express the whole number N is positional notation to base B 11

12 Let be B=10 Determine the largest integer k, such that B k N Specify how many digits will occur in the base B notation of the number N Determine the leading digit n k of the base B represented of N n k will be one of digits 0,1,..,B-1 largest integer n k B k N The next digit n k-1 is the largest integer such that n k-1 B k-1 N-n k B k And so on... 12

13 There is a tremendous gain in computational efficiency when two numbers expressed in positional notation are multiplied M=m k B k +...+m 1 B+m 0 N=n l B k +...+n 1 B+n 0 The product M*N is calculated by multiplying the single digits and adding the resulting columns, (k+1)(l+1) elementary operations instead of MN Example of a Taxonomy In 1887 Professor Harry Govier Seeley grouped all dinosaurs into the saurischia and ornithischia groups according to their hip design The saurischian were divided later into two subgroups: the carnivorous, bipedal theropods and the plant-eating, mostly quadruped sauropodomorphs The ornithischians were divided into the subgroups birdlike ornithopods, armored thyreophorans, and margginoncephalia The subgroups can be divided into suborders and then into families and finally into genus The genus includes the species 13

14 14

15 Another Example 15

16 Cost of Information Combination of the amount of information it produces and the effort required to obtain it Maximization of the information gained for a given effort invested (minimize the effort) Hierarchical structures that are characterized by the value of certain structural parameters If one assumes that time required for each elementary operation performed by the hierarchical organized information processing system is independent of the operation, then the time required to provide a unit of information will be a good measure of cost If the parameters of the system are selected so that the corresponding structure minimize the processing time, the system will be optimal for some parameters 16

17 Let be x 1,..,x n the parameters of the system We have some cost of the system (effort) E(x 1,..,x n ) and the corresponding produced information I(x 1,..,x n ) Then the solution to determine an optimal information-processing structure corresponds to maximize I/E Solved by calculus of variations Second law of Thermodynamics A physical system will extremize information for a given value of the energy by evolving towards a canonical distribution of its states which maximizes entropy and minimize the information 17

18 ..addition Example Range query Range query: search covers all points in the space whose Euclidian distance to the query y is smaller or equal to ε 18

19 DB[y] σ = y = Curse of dimensionality The metric indexes trees operate efficiently when the number of dimensions is small The growth of the number of dimensions has negative implications for the performance failing with the dimensionality eventually reducing the search time to sequential scanning problems arise from the fact that the volume of a sphere constant radius grows exponentially with increasing dimension 19

20 Linear subspace sequence Sequence of subspaces with, V=U 0 and Lower bounding lemma, Example, DB in subspace 20

21 Computing costs U={(x 1,x 2 ) R 2 x 1 =x 2 } 21

22 Orthogonal projection Corresponds to the mean value of the projected points Distance d between projected points in R m corresponds to the distance d u in the orthogonal subspace U multiplied by a constant c 22

23 Hierarchy of subspaces The distance between objects d=d U0 in the space U 0 can be obtained from the distance d Uk between objects in the orthogonal subspace U k by multiplying the distance d Uk by a constant Mean Euclidian distance for a query y to the elements of DB 23

24 Mean computing costs using the hierarchical subspace method Error bars indicate the standard deviation The x-axis indicates the number of the most similar images which are retrieved and the y-axis, the computing costs Computing costs simplification to: 24

25 25