CHAPTER-3 A New Spectral Technique Using Normalized Adjacency Matrices for Graph Matching Graph matching problem has found many applications in areas as diverse as chemical structure analysis, pattern recognition and bioinformatics. Many works have been reported for matching of specialized types of graphs. A new spectral technique based on eigenvalues and minimum eigenvector of the normalized adjacency matrix of the graphs is proposed in this chapter. The adjacency matrix representation of the graph is combined with the explicit information about the degree of each vertex to generate the normalized adjacency matrix of the graph [Hogben, 25]. The spectral properties of this matrix and the shortest distance sum from a vertex to other vertices are employed to match graphs and verify graph similarity/ isomorphism. Further the correspondence of the vertices between graphs is established by matching the values in the minimum eigenvectors of the normalized adjacency matrices. 3. Introduction In literature different matrix representations of the graphs are often used for finding the spectral properties. The matrix representations, which are employed for the purpose are, the Adjacency matrices, Laplacian matrices, the Signless Laplacian matrices and their normalized forms. In this work the normalized adjacency matrix representation of the graph and its spectral characteristics are employed for matching of the graphs along with the shortest distance sum to other vertices and vertex degrees. Further the eigenvalues and the minimum eigenvectors of such matrices representing the two graphs are used for finding vertex correspondence. Parts of this chapter appear in the paper Spectral Technique using Normalized Adjacency Matrices for Graph Matching published in International Journal of Computational Science and Mathematics, ISSN 974-389, Vol. 3 No. 4 (2), pp. 37-378. 52
The spectra of various matrices (i.e., the eigenvalues of the matrices) are used to obtain information about a graph. Spectral graph theory deals with exploration of graphs and study of their properties in relationship to the characteristic polynomial, eigenvalues, and eigenvectors of matrices associated with the graph, such as its adjacency matrix or laplacian matrix or their normalized forms. Two graphs are called isospectral or cospectral if the representing matrices of the graphs have equal multisets of eigenvalues [W]. The characteristic graph spectral properties of the various matrix representations are described in the following. The most common matrices that have been studied for simple undirected and unweighted graphs are; The Adjacency Matrix The Laplacian (Combinatorial) Matrix The Normalized Laplacian The Signless Laplacian The Normalized Adjacency Matrix 3.. The Adjacency Matrix Let G be a simple graph, one can define a matrix X(G), called the adjacency matrix, This representation is for undirected graph which implies that the adjacency matrix is symmetric. The spectrum of such a graph includes a set of eigenvalues which may be repeated and real. Such a matrix has a full set of eigenvectors which are mutually orthogonal. 53
The adjacency matrix uses s and s. This is useful for emphasizing the discrete nature of the graph. Other values can be assigned if it is a weighted graph. Some other spectral properties of the adjacency matrix representation of undirected graphs are listed below; The sum of the eigenvalues is the trace of the adjacency matrix which is since X(A) has for all elements on the diagonal. The sum of the square of the eigenvalues is the same as the trace of X(A 2 ). The diagonal entries of X(A 2 ) counts the number of closed walks of length 2, for which each edge is counted exactly twice The sum of the eigenvalues cubed is the same as the trace of X(A 3 ) i.e., the same as the number of closed walks of length 3. In adjacency matrix, eigenvector if and only if the graph is regular. is an 3..2 The Laplacian Matrix (Combinatorial Matrix) The Laplacian matrix L, is also called the admittance matrix or Kirchhoff matrix or combinatorial, of a graph G, and is defined for an undirected, unweighted graph without loops or multiple edges (i.e. Simple Graph). Let V be the vertex set and E be the edge set of graph G, if X(G) is a equation 3.[W2] symmetric adjacency matrix then L is defined by (3.) Where is the diagonal degree matrix, which is the diagonal matrix formed from the vertex degrees and X(G) is the adjacency matrix. The elements of L can be obtained as given in equation 3.2; (3.2) 54
A few spectral properties of the Laplacian Matrix are listed below, One special eigenvalue is zero. The fact is that the sum of all the rows is zero for zero eigenvalues. All the other eigenvalues are nonnegative, so the combinatorial Laplacian is positive semi-definite. For a graph G and its Laplacian matrix L with eigenvalues L is always positive semi-definite. Relation between Laplacian matrix L and incidence matrix M : If we define an oriented incidence matrix M with element M ev for edge e (connecting vertex i and j, with i < j) and vertex v is given by equation 3.3. (3.3) Then the laplacian matrix L satisfies the realtion, Where is the matrix transpose of The second smallest eigenvalue of the Laplacian matrix of G is the algebraic connectivity of G. This eigenvalue is named the algebraic connectivity of the graph because it serves as a lower bound on the degree of robustness of the graph to node and edge failures. This follows from the following inequality where, ν(g) is the node-connectivity and η (G) is the edge-connectivity of a graph. Therefore, a network with high algebraic connectivity is robust to both node and edge failures. The algebraic connectivity of a graph G is greater than if and only if G is a connected graph. The value of the algebraic connectivity is bounded above by the vertex connectivity of the graph. The algebraic connectivity is bounded below 55
by, where n is the number of vertices of a connected graph and D is the diameter and in fact in a result due to Brendan McKay by [W6]. Unlike the traditional connectivity, the algebraic connectivity is dependent on the number of vertices, as well as the way in which vertices are connected. In random graphs, the algebraic connectivity decreases with the number of vertices, and increases with the average degree. The number of times zero appears as an eigenvalue in the Laplacian gives the number of connected components in the graph. The smallest non-zero eigenvalue of L is called the spectral gap. These are some important spectral properties of the Laplacian matrix. 3..3 The Normalized Laplacian Matrix A normalized version of the Laplacian matrix, denoted by equation 3.4 [ Hall, 2] is formally defined as in if i jand d L i, j - if i and j are adjacent (3.4) d d i j otherwise j For a graph G with no isolated vertices the relationship between normalized Laplacian and the Laplacian matrix L is given by equation 3.5 Some of the spectral properties of the normalized laplacian are enlisted below, Zero is an eigenvalue and the remaining eigenvalues are nonnegative. (3.5) 56
The difference between the spectrums of the laplacian and the normalized laplacian is that; for the combinatorial laplacian the eigenvalues can be essentially large, whereas the normalized laplacian has eigenvalues always lying in the range between zero and two. 3..4 The Signless Laplacian Matrix The Signless Laplacian matrix of a simple graph G is defined as in equation 3.6 (3.6) X(G) is a symmetric adjacency matrix and D is the diagonal matrix of vertex degrees 3..5 Normalized Adjacency Matrix The normalized adjacency matrix N(G) of graph is a unique representation that combines the degree information of each vertex and their adjacency information in the graph. The normalized adjacency matrix is obtained from the adjacency matrix of the graph. Consider an undirected graph G = (V, E) where V is the set of vertices, E is the set of edges and X(G) is the adjacency matrix representing G. The diagonal degree matrix, represents the degrees of each vertex of the graph. The normalized adjacency matrix is given by equation 3.7, N ( G) D X ( G) D, where diag( degg,... deg n). D G (3.7) N (G) is referred to as normalized adjacency matrix [W3]. The normalized adjacency matrix is defined only for graphs without isolated vertices. This is because the degree of the isolated vertex is zero and cannot be uniquely determined for such graphs. Hence the proposed methodology is suitable only for graphs without isolated vertices. 57
Some common spectral properties of various matrices are listed below; For an undirected graph G, all the eigenvalues of the adjacency matrix are zero if and only if G has no edges. The same holds for the Laplace eigenvalues and the signless Laplace eigenvalues. If d is a diameter of a connected graph G. Then G has at least distinct eigenvalues for an adjacency matrix. The same holds for the Laplace eigenvalues and the signless Laplace eigenvalues. Let G be a graph with connected components. Then the spectrum of G is the union of the spectra of (and multiplicities are added) for the adjacency matrix. The same holds for the Laplace and the signless Laplace spectrum. For an undirected, regular graph G of degree k. The largest eigenvalue of G is k, and its multiplicity equals the number of connected components of G in an adjacency matrix. The spectrum of J is n (with multiplicity ), zero (with multiplicity n-). Where J is all- matrix of order n and rank, and that the all- vector is an eigenvector with eigenvalue n. The spectrum of adjacency matrix A is (n ) (with multiplicity ), ( ) (with multiplicity (n-), where is an adjacency matrix of the complete graph on n vertices of graph G. The Laplace matrix is, which has spectrum zero (with multiplicity ), n (with multiplicity n-). For an undirected graph G the multiplicity of zero as Laplace eigenvalues, equals the number of connected components of G. A graph G is bipartite if and only if the Laplace spectrum and the signless Laplace spectrum of G are equal. 58
In this work the spectral property correspondence of graphs, along with shortest distance sum and degree invariance, has been used to prove the similarity of graphs (for graph matching). The remaining part of the chapter describes the methodology employed for finding whether two graphs are same/ similar/ isomorphic. 3..6 Overview of the Proposed Methodology The proposed methodology involves finding the normalized adjacency matrix representation of the two graphs and computing the eigenvalues and eigenvectors of the same. Further to find isomorphism, the degree and the shortest distance from a vertex to all the other vertices in the graph are computed. If the degree and the shortest distances sum of the two graphs are equivalent and the eigenvalues of the normalized adjacency matrices of the two graphs are the same, then the two graphs are similar/ isomorphic. Further the vertex correspondence is obtained using the equivalent values of minimum eigenvector of the normalized adjacency matrix. The methodology has been implemented using MATLAB and tested on a large number of graphs. The results are excellent. The complete methodology is described in section 3.2 3.2 The Proposed Methodology Graph matching involves finding out whether two graphs are same or similar or isomorphic. Two graphs are said to be isomorphic if there is a correspondence between their vertex sets such that they also preserve adjacency. Thus is isomorphic to if there is a bijection (3.8) Clearly, isomorphic graphs have same number of vertices and edges. The diagonal degree matrix and its inverse are employed to find normalized adjacency matrix from the adjacency matrix of the graph as depicted in equation 3.7. If the 59
eigenvalues of the normalized adjacency matrices of the two graphs have a one to one correspondence and the shortest distance sums along with vertex degrees match then the two graphs are isomorphic. Further the eigenvectors corresponding to minimum eigenvalues of the two normalized adjacency matrices are employed for finding the mapping between the vertices of the two graphs. The eigenvalues of the normalized adjacency matrices are known to be invariant for isomorphic transformations. Hence eigenvalues of the normalized adjacency matrices of two isomorphic/ similar graphs are required to be the same. Further based on theorem 2. the following observation 3. of the spectral property of normalized adjacency matrices along with the invariance of the vertex degree and shortest distance sum of vertex with other vertices can be made. Observation 3. The invariance of vertex degree, shortest distance sum of vertices and eigenvalues of normalized adjacency matrix representation of the two graphs are sufficient conditions for graphs to be isomorphic Let N (G ), be the normalized adjacency matrix of G and N 2 (G 2 ) be the normalized adjacency matrix of the graph G 2. The spectral properties of the graphs are found by computing the eigenvalues and eigenvectors of N and N 2. Let E =Eigenvalue (N ) and E 2 =Eigenvalue (N 2 ), be the eigenvalues of the normalized adjacency matrices N and N 2. If (all the elements of E are also present in E 2 ) then the two graphs G and G 2 are candidates for similarity/ isomorphism. The isomorphism of the graphs G and G 2 is ascertained if and only if; there is equivalence of shortest distance sums and degrees of corresponding vertices between the two graphs. 6
Further to find vertex/ node correspondences between two isomorphic graphs, the minimum eigenvectors are employed. Let EV =Eigenvector(N ) and EV 2 =Eigenvector(N 2 ) be the eigenvectors of N and N 2. The correspondence of the vertices is found by matching the values of the eigenvectors corresponding to the minimum eigenvalues of G and G 2. If the matching results in less than matches (empirical value) one of the minimum eigenvector is negated and further matching is obtained and this gives the correspondence between the vertices of the two graphs. This process is based on the property of the graph spectra given as Observation 3.2. The complete process of the graph matching is depicted in the flow chart given in Figure 3.. The same is explained using an example in section 3.2.. Observation 3.2: The matching of the absolute values of the coefficients of minimum eigenvector of the normalized adjacency matrix of two isomorphic graphs, gives a possible correspondence between vertices of the two graphs. 6
Start Read the edge list and generate adjacency matrices X (G ) and X (G 2 ) Find the diagonal matrices of the two graphs (D,D 2 ), D= diag(d,d 2, d n ) & shortest distance sums(sp,sp 2 ) Find the normalized adjacency matrix N and N 2 using the formula D X ( G) D Flag check for shortest path & degree match bet n vertices of G and G 2, if so assign else assign Find eigenvalues of N and N 2, E = Eig(N ) and E 2 =Eig(N 2 ) Yes If E ΞE 2 & flag A No Print graphs are not isomorphic stop Figure 3.: Flow chart for graph matching using normalized adjacency matrices (continued ) 62
A EV = Min eigenvector (N ) EV 2 = Min eigenvector (N 2 ) i= i=i+ j= i=i+ No Yes No i n No Yes J > n EV (i)=ev 2 ( j) No Yes Map (k,) Map (k,2) k k+ i j i n Yes j j+ Print The graphs are isomorphic and MAP gives vertex correspondence No k < n/2 Yes EV2 = - EV2 Stop Figure 3.: Flow chart for graph matching using normalized adjacency matrices. 63
3.2. An Example The methodology for matching graphs using eigenvalues and eigenvectors of the normalized adjacency matrix is described using an example in this sub section. The example considers two six vertex graphs. The two six vertex graphs G and G 2 are shown in Figure 3.2 below V V V 5 V 6 V 2 V 5 V 3 V 2 V 3 V 6 V 4 V 4 Graph G Graph G 2 Figure 3.2: Graphs G and G 2 The adjacency matrices of G and G 2 are given below. X(G ) X(G 2 ) The normalized adjacency matrices N and N 2 are computed and given below. 64
.3536.3536.2887.5.3536.482.3536.5 N =NAM(G ) =.5.482.2887.482.482.5.3536.3536.5.2887.5.482 N 2 =NAM(G 2 ) =.3536.3536.5.482.5.2887.482.482 The eigenvalues of the two graphs computed from N and N 2 and are shown below; E = -.8567 -.576 -.243.245.42. E 2 = -.8567 -.576 -.243.245.42. The eigenvectors of the graphs are again computed from N and N 2 and are shown below; 65
EV = -.53.4329 -.235.2259 -.3866.5345.334 -.584 -.42 -.524 -.258.378.4958.433 -.4853.445.494.378 -.4747 -.3559.3589.583.66.378.3889.4492.459 -.5.878.4629.393 -.3757.498.46 -.488.2673 EV 2 = -.53 -.4747.4958.334.393.3889.4329 -.3559.433 -.584 -.3757.4492 -.235.3589 -.4853 -.42.498.459 -.2259 -.583 -.445.524 -.46.5.3866 -.66 -.494.258.488 -.878 -.5345 -.378 -.378 -.378 -.2673 -.4629 The Table 3. below lists the parameters degrees, shortest distance sum of the vertex and absolute eigenvalues, the Sl No also represents the vertex number for graphs G and G 2. Table: 3.: Eigenvalues, degrees and average shortest distance of the vertices Sl.No. Degree of Vertices in G Shortest distance sum with other vertices of G Degree of Vertices in G 2 Shortest distance sum with other vertices of G 2 Eigen values of graph G Eigen values of graph G 2 4 6 4 6.8567.8567 2 2 8 2 9.576.576 3 2 8 2 8.243.243 4 2 9 2 8.245.245 5 3 7.42.42 6 3 7.. From the table 3., one can see that the shortest distance sums and degrees are equivalent (ie. the shortest distance sum and degrees with same multiplicities are found in both graphs). Further the eigenvalues are also equivalent; hence the graphs G and G 2 are isomorphic. The vertex correspondence between the two graphs is established using minimum eigenvector of N and N 2.The minimum eigenvectors of N and N 2, namely MinEig and Min Eig2 are enlisted below. 66
MinEig= [-.53.334.4958 -.4747.3889.393] MinEig2= [-.53 -.4747.4958.334.393.3889] Mapping of the vertices are obtained from the above vectors and are as shown below. Vertex Correspondence = 2 3 4 5 6 4 3 2 6 5 Hence vertex correspondence between graphs G and G 2 are [-, 2-4, 3-3, 4-2, 5-6, 6-5], which is true by observation. 3.2.2 Comments on Correctness of the Methodology In this chapter, it has been shown that the two graphs are similar/ isomorphic, if the graphs have same number of vertices, degrees, and eigenvalues along with the shortest distance sum from a vertex to all the other vertices. A few examples showing that the graphs having the same spectra (i.e. they are cospectral) but are non-isomorphic graphs are enunciated in the following. The proposed methodology uses the following three properties for establishing vertex correspondence viz: i) Cospectral normalized adjacency matrices i.e., NAM S of two graphs N and N 2 have same eigenvalues. ii) Equivalent diagonal degree matrices. The diagonal degree matrices D and D 2 of both the graphs have equal number of similar degreed vertices. 67
iii) Equivalent shortest distance sum of the two graphs. The vertices with same degree have the same shortest distance sum to other vertices in both the graphs. The methodology has been tested on large number of synthetic graph pairs and accurate results have been obtained. It is known that some non-isomorphic graphs are cospectral [Dam and Haemers 23]. In the methodology along with cospectral nature of the normalized adjacency matrices of the graphs the two other characteristic features namely equivalent degrees and shortest distance sums are used. These characteristics filter out cospectral non-isomorphic graphs. For example the graphs in Figure 3.3 are cospectral and are of the same number of vertices and equivalent degrees as both of them are r-regular (r=4) graphs. They do not satisfy the equivalent shortest distance characteristic and hence are not isomorphic according to the methodology, which is true. Graph G Graph G 2 Figure 3.3: Two cospectral regular graphs G & G 2 which are non isomorphic The graph pair in figure 3.4 are also cospectral but non-isomorphic. These graphs are the smallest pair of graphs which are co-spectral. The proposed methodology correctly identifies them as non-isomorphic as they do not have equivalent degrees. 68
Figure 3.4: Cospectral smallest graphs The same characteristic requirement also identifies the pair of graphs in figure 3.5 as nonisomorphic. The two graphs in figure 3.5 are cospectral but are non-isomorphic as is apparent by the appearance of the two graphs. One of the graphs in figure 3.5 has a vertex with degree 5 whereas the other does not. Figure 3.5: Two non-isomorphic cospectral graphs that does not satisfy degree invariance Many such counter example graphs have been tested for matching and the results obtained are as expected. Typical graphs for isomorphism testing enlisted in chapter 2, have also obtained accurate results. Basically the shortest distance sum catches all the typically non-isomorphic graphs which look or seem isomorphic and hence is a powerful and sufficient characteristic for verifying graph matching as discussed in chapter 2. Hence the methodology proposed in this chapter is correct for undirected simple graphs. The experimentation conducted with this methodology is described in section 3.3. 69
3.3 Experimentation The proposed methodology for the graph matching is implemented using MATLAB 7. The methodology has been tested on one hundred and twenty five synthetic graph pairs and % accurate results have been achieved. Among these seventy four pairs were isomorphic and the remaining graph pairs were non isomorphic. The test results for a few of the different graphs having different number of vertices have been enlisted in Table 3.2, along with the time taken by the methodology. Table 3.2: The results of graph matching using normalized adjacency matrices Sl No. of No. of Isomorphic Correct Mapping of Time for No Vertices Edges (Y/ N) Vertices (Y/ N) Computation 4 4 Y Y.938 2 5 6 Y Y.875 3 5 4 Y Y.625 4 5 5 Y Y.46 5 5 6 N - - 6 6 6 Y Y.25 7 6 7 Y Y.2344 8 6 7 Y Y.3594 9 6 7 N - - 6 6 Y Y.563 7 9 Y Y.3438 2 7 N - - 3 8 8 Y Y.396 4 8 8 Y Y.556 5 8 9 N - - 6 8 8 Y Y.453 7 8 7 N - - 8 8 9 Y Y.5 9 9 Y Y.679 2 9 Y Y.4375 2 9 Y Y.694 22 9 4 N - - 23 2 Y Y.33 24 2 Y Y.75 25 N - - 26 Y Y.5938 27 3 N - - 7
Time in Seconds 28 3 Y Y.828 29 3 Y Y.963 3 3 Y Y.56 3 3 Y Y.288 32 3 N - - 33 4 3 Y Y 2.56 34 5 5 N - - 35 2 6 Y Y 5.9688 36 22 24 Y Y 6.953 37 24 27 Y Y.283 38 26 32 Y Y 3.6875 39 28 36 Y Y 7.4375 4 3 4 Y Y 22.283 The time taken by the methodology v/s the number of vertices of the graphs is plotted in Figure 3.6. 25 Computation Time 2 5 5 Computation Time 2 3 4 Number of nodes Figure 3.6: Time v/s Number of Vertices Plot The nature of the graph shows an increase in time with increase in the number of vertices. The time complexity of the algorithm is found to be polynomial and detailed analysis is provided in Chapter 6. 7
3.4 Summary In the present work a new spectral technique using normalized adjacency matrices for graph matching is presented. The method makes use of eigenvalues of the two normalized adjacency matrices of the graphs. If the vertices of the two graphs have the same eigenvalues then the two graphs are assumed to be candidates for similarity/isomorphism. The similarity is ascertained by checking for the equivalence of degrees and shortest distance sum. Further, if the graphs are found to be isomorphic/ similar, the eigenvectors corresponding to minimum eigenvalues are employed for finding the correspondence between the vertices of the graph. The program developed in MATLAB 7, is tested using about hundred and twenty five pairs of synthetic graphs with different numbers of vertices and accurate results have been achieved. This methodology has proved to be correct and efficient for a large number of graphs. Further the use of normalized adjacency matrix representation of the graph has proven to be beneficial as it contains not only the adjacency information but the degree information of each vertex stated explicitly. This representation is also simple for computation purpose and holds a lot of promise for other applications. 72