Fastest Mixing Markov Chain on Graphs with Symmetries

Transcription

1 Fastest Mixig Markov Chai o Graphs with Symmetries Stephe Boyd Persi Diacois Pablo A Parrilo Li Xiao December 5, 2006 Abstract We show how to exploit symmetries of a graph to efficietly compute the fastest mixig Markov chai o the graph (choose the trasitio probabilities o the edges to miimize the secod-largest eigevalue modulus of the trasitio probability matrix) Exploitig symmetry ca lead to sigificat reductio i both umber of optimizatio variables ad size of matrices i solvig the correspodig semidefiite programs The problem ca be cosiderably simplified ad is ofte solvable by oly exploitig symmetry We obtai aalytic or semiaalytic results for particular classes of graphs, such as edge-trasitive ad distace-trasitive graphs We describe two geeral approaches for symmetry exploitatio based o orbit theory ad block-diagoalizatio, respectively These approaches eable umerical solutio of largescale istaces, otherwise computatioally ifeasible to solve 1 Itroductio I the fastest mixig Markov chai problem, we choose the trasitio probabilities o the edges of a graph to miimize the secod-largest eigevalue modulus of the trasitio probability matrix I [BDX04] we formulated this problem as a covex optimizatio problem, i particular as a semidefiite program Thus it ca be solved, up to ay give precisio, i polyomial time by iterior-poit methods I this paper, we show how to exploit symmetries of a graph to make the computatio more efficiet 11 The fastest mixig Markov chai problem We cosider a udirected graph G = (V, E) with vertex set V = {1,,} ad edge set E ad assume that G is coected We defie a discrete-time Markov chai o the vertices as follows The state at time t will be deoted X(t) V, for t = 0, 1, Each edge i the graph is associated with a trasitio probability with which X makes a trasitio betwee the two adjacet vertices This Markov chai ca be described via its trasitio probability matrix P R, where P ij = Prob ( X(t + 1) = j X(t) = i ), i, j = 1,, Note that P ii is the probability that X(t) stays at vertex i, ad P ij = 0 for {i, j} / E (trasitios are allowed oly betwee vertices that are liked by a edge) We assume that the trasitio Departmet of Electrical Egieerig, Staford Uiversity, Staford, CA boyd@stafordedu Departmet of Statistics ad Departmet of Mathematics, Staford Uiversity, Staford, CA Departmet of Electrical Egieerig ad Computer Sciece, Massachusetts Istitute of Techology, Cambridge, MA parrilo@mitedu Microsoft Research, 1 Microsoft Way, Redmod, WA lixiao@microsoftcom 1

2 probabilities are symmetric, ie, P = P T, where the superscript T deotes the traspose of a matrix Of course this trasitio probability matrix must also be stochastic: P 0, P1 = 1, where the iequality P 0 meas elemetwise, ad 1 deotes the vector of all oes Sice P is symmetric ad stochastic, the uiform distributio (1/)1 T is statioary I additio, the eigevalues of P are real, ad o more tha oe i modulus We deote them i o-icreasig order 1 = λ 1 (P) λ 2 (P) λ (P) 1 We deote by µ(p) the secod-largest eigevalue modulus (SLEM) of P, ie, µ(p) = max i=2,, λ i(p) max {λ 2 (P), λ (P)} This quatity is widely used to boud the asymptotic covergece rate of the distributio of the Markov chai to its statioary distributio, i the total variatio distace or chi-squared distace (eg, [DS91, DSC93]) I geeral the smaller µ(p) is, the faster the Markov chai coverges For more backgroud o Markov chais, eigevalues ad rapid mixig, see, eg, the text [Bré99] I [BDX04], we addressed the followig problem: What choice of P miimizes µ(p)? I other words, what is the fastest mixig (symmetric) Markov chai o the graph? This ca be posed as the followig optimizatio problem: miimize µ(p) subject to P 0, P1 = 1, P = P T P ij = 0, {i, j} / E (1) Here P is the optimizatio variable, ad the graph is the problem data We call this problem the fastest mixig Markov chai (FMMC) problem This is a covex optimizatio problem, i particular, the objective fuctio ca be explicitly writte i a covex form µ(p) = P (1/)11 T 2, where 2 deotes the spectral orm of a matrix Moreover, this problem ca be readily trasformed ito a semidefiite program (SDP): miimize s subject to si P (1/)11 T si P 0, P1 = 1, P = P T P ij = 0, {i, j} / E (2) Here I deote the idetity matrix, ad the variables are the matrix P ad the scalar s The symbol deotes matrix iequality, ie, X Y meas Y X is positive semidefiite There has bee some follow-up work o this problem Boyd, Diacois, Su, Xiao ([BDSX06]) proved aalytically that o a -path the fastest mixig chai ca be obtaied by assigig the same trasitio probability half at the 1 edges ad two loops at the two eds Roch ([Roc05]) used stadard mixig-time aalysis techiques (variatioal characterizatios, coductace, caoical paths) to boud the fastest mixig time Gade ad Overto ([GO06]) have cosidered the fastest mixig problem for a oreversible Markov chai Here, the problem is o-covex ad much remais to be doe Fially, closed form solutios of fastest mixig problems have recetly bee applid i statistics to give a geeralizatio of the usual spectral aalysis of time series for more geeral discrete data see [Sal06] 2

3 12 Exploitig problem structure The SDP formulatio (2) meas that the FMMC problem ca be efficietly solved usig stadard SDP solvers, at least for small or medium size problems (with umber of edges up to a thousad or so) Geeral backgroud o covex optimizatio ad SDP ca be foud i, eg, [NN94, VB96, WSV00, BTN01, BV04] The curret SDP solvers (eg, [Stu99, TTT99, YFK03]) mostly use iterior-poit methods which have polyomial time worst-case complexity Whe solvig the SDP (2) by iterior-poit methods, i each iteratio we eed to compute the first ad secod derivatives of the logarithmic barrier fuctios (or potetial fuctios) for the matrix iequalities, ad assemble ad solve a liear system of equatios (the Newto system) Let be the umber of vertices ad m be the umber of edges i the graph (equivaletly m is the umber of variables i the problem) The Newto system is a set of m liear equatios with m ukows Without exploitig ay structure, the umber of flops per iteratio i a typical barrier method is o the order max{m 3, m 2 2, m 3 }, where the first two terms come from computig ad assemblig the Newto system, ad the third term amouts to solvig it (see, eg, [BV04, 1183]) (Other variats of iterior-poit methods have similar orders of flop cout) Exploitig problem structure ca lead to sigificat improvemet of solutio efficiecy As for may other problems defied o a graph, sparsity is the most obvious structure to cosider here I fact, may curret SDP solvers already exploit sparsity However, as a well-kow fact, exploitig sparsity aloe i iterior-poit methods for SDP has limited effectiveess The sparsity of P, ad the sparsity plus rak-oe structure of P (1/)11 T, ca be exploited to sigificatly reduce the complexity of assemblig the Newto system, but typically the Newto system itself is dese The computatioal cost per iteratio ca be reduced to order O(m 3 ), domiated by solvig the dese liear system (see aalysis for similar problems i, eg, [BYZ00, XB04, XBK07]) I additio to usig iterior-poit methods for the SDP formulatio (2), we ca also solve the FMMC problem i the form (1) by subgradiet-type (first-order) methods The subgradiets of µ(p) ca be obtaied by computig the extreme eigevalues ad associated eigevectors of the matrix P This ca be doe very efficietly by iterative methods, specifically the Laczos method, for large sparse symmetric matrices (eg, [GL96, Saa92]) Compared with iterior-poit methods, subgradiet-type methods ca solve much larger problems but oly to a moderate accuracy (they do t have polyomial-time worst-case complexity) I [BDX04], we used a simple subgradiet method to solve the FMMC problem o graphs with up to a few hudred thousad edges More sophisticated subgradiet-type methods for solvig large-scale eigevalue optimizatio problems ad SDPs have bee reported i, eg, [HR00, Nem04, LNM04, Nes05] A successive partial liear programmig method was developed i [Ove92] I this paper, we focus o the FMMC problem o graphs with large symmetry groups, ad show how to exploit symmetries of the graph to make the computatio more efficiet A result by Erdős ad Réyi [ER63] states that with probability oe, the symmetry group of a (suitably defied) radom graph is trivial, ie, it cotais oly the idetity elemet Nevertheless, may of the graphs of theoretical ad practical iterest, particularly i egieerig applicatios have very iterestig, ad sometimes very large, symmetry groups Symmetry reductio techiques have bee explored i several differet cotexts, eg, dyamical systems ad bifurcatio theory [GSS88], polyomial system solvig [Wor94, Gat00], umerical solutio of partial differetial equatios [FS92], ad Lie symmetry aalysis i geometric mechaics [MR99] I the cotext of optimizatio, symmetry has bee exploited to prue the eumeratio tree i brach-ad-cut algorithms for iteger programmig [Mar03] ad to reduce matrix size i a spectral radius optimizatio problem [HOY03]; A class of SDPs with symmetry has bee defied i [KOMK01], where the authors study the ivariace properties of the search directios of primal-dual iterior-poit methods 3

4 For the FMMC problem, we show that exploitig symmetry allows sigificat reductio i both umber of optimizatio variables ad size of matrices Effectively, they correspod to reducig m ad, respectively, i the flop couts for iterior-poit methods metioed above The problem ca be cosiderably simplified ad is ofte solvable by oly exploitig symmetry This leads to aalytic (or semi-aalytic) results for particular classes of graphs, such as edge- ad distacetrasitive graphs I additio, we preset two geeral approaches for symmetry exploitatio ad discuss their coectios Oe approach is based o the orbit theory developed i [BDPX05], i which we fid the fastest mixig reversible Markov chai o a (much smaller) orbit graph that cotais all the distict eigevalues of the origial Markov chai, but with much less multiplicities The other approach follows more geeral results o ivariat SDPs developed i [GP04], i which the trasitio probability matrix is block diagoalized by costructig a symmetry-adapted basis for the represetatios of the symmetry group These two geeral approaches eable umerical solutio of large-scale istaces, otherwise computatioally ifeasible to solve 13 Outlie I 2, we explai the cocepts of graph automorphisms ad the automorphism group (symmetry group) of a graph We show that the FMMC problem always attais its optimum i the fixed-poit subset of the feasible set uder the automorphism group This allows us to oly cosider a umber of distict trasitio probabilities that equals the umber of orbits of the edges We the give a formulatio of the FMMC problem with reduced umber of variables (trasitio probabilities), which appears to be very coveiet i subsequet sectios I 3, we give closed-form solutios for the FMMC problem o some special classes of graphs, amely edge-trasitive graphs ad distace-trasitive graphs Alog the way we also discuss FMMC o graphs formed by takig Cartesia products of simple graphs I 4, we first review the orbit theory for reversible Markov chais, ad give sufficiet coditios o costructig a orbit chai that cotai all distict eigevalues of the origial chai This orbit chai is usually o loger symmetric but always reversible We the solve the fastest reversible Markov chai problem o the orbit graph, from which we immediately obtai optimal solutio to the origial FMMC problem I 5, we review some group represetatio theory ad show how to block diagoalize the liear matrix iequalities i the FMMC problem by costructig a symmetry-adapted basis The resultig blocks usually have much smaller sizes ad repeated blocked ca be discarded i computatio Extesive examples i 4 ad 5 reveal iterestig coectios betwee these two geeral symmetry reductio methods I 6, we coclude the paper by poitig out some possible future work 2 Symmetry aalysis I this sectio we explai the basic cocepts that are essetial i exploitig graph symmetry, ad derive our result o reducig the umber of optimizatio variables i the FMMC problem 21 Graph automorphisms ad classes The study of graphs that possess particular kids of symmetry properties has a log history The basic object of study is the automorphism group of a graph, ad differet classes ca be defied depedig o the specific form i which the group acts o the vertices ad edges 4

5 Figure 1: The graph o the left side is edge-trasitive, but ot vertex-trasitive The oe o the right side is vertex-trasitive, but ot edge-trasitive A automorphism of a graph G = (V, E) is a permutatio σ of V such that {i, j} E if ad oly if {σ(i), σ(j)} E The (full) automorphism group of the graph, deoted by Aut(G), is the set of all such permutatios, with the group operatio beig compositio For istace, for the graph o the left i Figure 1, the correspodig automorphism group is geerated by all permutatios of the vertices {1, 2, 3} This group, isomorphic to the symmetric group S 3, has six elemets, amely the permutatios (the idetity), , , , , ad Note that vertex 4 caot be permuted with ay other vertex Recall that a actio of a group G o a set X is a homomorphism from G to the set of all permutatios of the elemets i X (ie, the symmetric group of degree X ) For a elemet x X, the set of all images g(x), as g varies through G, is called the orbit of x Distict orbits form equivalet classes ad they partitio the set X The actio is trasitive if for every pair of elemets x, y X, there is a group elemet g G such that g(x) = y I other words, the actio is trasitive if there is oly oe sigle orbit i X A graph G = (V, E) is said to be vertex-trasitive if Aut(G) acts trasitively o V The actio of a permutatio σ o V iduces a actio o E with the rule σ({i, j}) = {σ(i), σ(j)} A graph G is edge-trasitive if Aut(G) acts trasitively o E Graphs ca be edge-trasitive without beig vertex-trasitive ad vice versa; simple examples are show i Figure 1 A graph is 1-arc-trasitive if give ay four vertices u, v, x, y with {u, v}, {x, y} E, there exists a automorphism g Aut(G) such that g(u) = x ad g(v) = y Notice that, as opposed to edge-trasitivity, here the orderig of the vertices is importat, eve for udirected graphs I fact, a 1-arc trasitive graph must be both vertex-trasitive ad edge-trasitive, ad the reverse may ot be true The 1-arc-trasitive graphs are called symmetric graphs i [Big74], but the moder use exteds this term to all graphs that are simultaeously edge- ad vertex-trasitive Fially, let δ(u, v) deote the distace betwee two vertices u, v V A graph is called distace-trasitive if, for ay four vertices u, v, x, y with δ(u, v) = δ(x, y), there is a automorphism g Aut(G) such that g(u) = x ad g(v) = y The cotaimet relatioship amog the four classes of graphs described above is illustrated i Figure 2 Explicit couterexamples are kow for each of the o-iclusios It is geerally believed that distace-trasitive graphs have bee completely classified This work has bee doe by classifyig the distace-regular graphs It would take us too far afield to give a complete discussio See the survey i [DSC06, Sectio 7] The cocept of graph automorphism ca be aturally exteded to weighted graphs, by requirig that the permutatio must also preserve the weights o the edges (eg, [BDPX05]) This extesio allows us to exploit symmetry i more geeral reversible Markov chais, where the trasitio 5

6 Distace-trasitive Edge-trasitive 1-arc trasitive Vertex-trasitive Figure 2: Classes of symmetric graphs, ad their iclusio relatioship probability matrix is ot ecessarily symmetric 22 FMMC with symmetry costraits A permutatio σ Aut(G) ca be represeted by a permutatio matrix Q, where Q ij = 1 if i = σ(j) ad Q ij = 0 otherwise The permutatio σ iduces a actio o the trasitio probability matrix by σ(p) = QPQ T We deote the feasible set of the FMMC problem (1) by C, ie, C = {P R P 0, P1 = 1, P = P T, P ij = 0 for {i, j} / E} This set is ivariat uder the actio of graph automorphism To see this, let h = σ(i) ad k = σ(j) The we have (σ(p)) hk = (QPQ T ) hk = l (QP) hl Q kl = (QP) hj = l Q hl P lj = P ij Sice σ is a graph automorphism, we have {h, k} E if ad oly if {i, j} E, so the sparsity patter of the probability trasitio matrix is preserved It is straightforward to verify that the coditios P 0, P1 = 1, ad P = P T, are also preserved uder this actio Let F deote the fixed-poit subset of C uder the actio of Aut(G); ie, F = {P C σ(p) = P, σ Aut(G)} We have the followig theorem (see also [GP04, Theorem 33]) Theorem 21 The FMMC problem always has a optimal solutio i the fixed-poit subset F Proof Let µ deote the optimal value of the FMMC problem (1), ie, µ = if{µ(p) P C} Sice the objective fuctio µ is cotiuous ad the feasible set C is compact, there is at least oe optimal trasitio matrix P such that µ(p ) = µ Let P deote the average over the orbit of P uder Aut(G) 1 P = σ(p ) Aut(G) σ Aut(G) This matrix is feasible because each σ(p ) is feasible ad the feasible set is covex By costructio, it is also ivariat uder the actios of Aut(G) Moreover, usig the covexity of µ, we have µ(p) µ(p ) It follows that P F ad µ(p) = µ 6

7 As a result of Theorem 21, we ca replace the costrait P C by P F i the FMMC problem ad get the same optimal value I the fixed-poit subset F, the trasitio probabilities o the edges withi a orbit must be the same So we have the followig corollaries: Corollary 22 The umber of distict edge trasitio probabilities we eed to cosider i the FMMC problem is at most equal to the umber of orbits of E uder Aut(G) Corollary 23 If G is edge-trasitive, the all the edge trasitio probabilities ca be assiged the same value Note that the holdig probability at the vertices ca always be elimiated usig P ii = 1 j P ij So it suffices to oly cosider the edge trasitio probabilities 23 Formulatio with reduced umber of variables From the results of the previous sectio, we ca reduce the umber of optimizatio variables i the FMMC problem from the umber of edges to the umber of edge orbits uder the automorphism group Here we give a explicit parametrizatio of the FMMC problem with the reduced umber of variables This parametrizatio is also the precise characterizatio of the fixed-poit subset F Recall that the adjacecy matrix of a graph with vertices is a matrix A whose etries are give by A ij = 1 if {i, j} E ad A ij = 0 otherwise Let ν i be the valecy (degree) of vertex i The Laplacia matrix of the graph is give by L = Diag(ν 1, ν 2,,ν ) A, where Diag(ν) deotes a diagoal matrix with the vector ν as its diagoal Extesive accout of the Laplacia matrix ad its use i algebraic graph theory are provided i, eg, [Mer94, Chu97, GR01] Suppose that there are N orbits of edges uder the actio of Aut(G) For each orbit, we defie a orbit graph G k = (V, E k ), where E k is the set of edges i the kth orbit Note that the orbit graphs are discoected (there are discoected vertices) if the origial graph is ot edge-trasitive Let L k be the Laplacia matrix of G k Note that the diagoal etries (L k ) ii equals the valecy of ode i i G k (which is zero if vertex i is discoected with all other vertices i G k ) By Corollary 22, we ca assig the same trasitio probability o all the edges i the k-th orbit Deote this trasitio probability by p k ad let p = (p 1,,p N ) The the trasitio probability matrix ca be writte as N P(p) = I p k L k (3) This parametrizatio of the trasitio probability matrix automatically satisfies the costraits P = P T, P1 = 1, ad P ij = 0 for {i, j} E The etry-wise oegative costrait P 0 ow traslates ito k=1 p k 0, k = 1,,N N (L k ) ii p k 1, i = 1,, k=1 where the first set of costraits are for the off-diagoal etries of P, ad the secod set of costraits are for the diagoal etries of P It ca be verified that the parametrizatio (3), together with the above iequality costraits, is the precise characterizatio of the fixed-poit subset F Therefore we ca explicitly write the 7

8 FMMC problem restricted to the fixed-poit subset as ( miimize µ I ) N k=1 p kl k subject to p k 0, k = 1,,N N k=1 (L k) ii p k 1, i = 1,, (4) Later i this paper, we will also eed the correspodig SDP formulatio miimize s subject to si I N k=1 p kl k (1/)11 T si p k 0, k = 1,,N N k=1 (L k) ii p k 1, i = 1,, (5) 3 Some aalytic results For some special classes of graphs, the FMMC problem ca be cosiderably simplified ad ofte solved by oly exploitig symmetry I this sectio, we give some aalytic results for the FMMC problem o edge-trasitive graphs, Cartesia product of simple graphs, ad distace-trasitive graphs (a subclass of edge-trasitive graphs) The optimal solutio is ofte expressed i terms of the eigevalues of the adjacecy matrix or the Laplacia matrix of the graph It is iterestig to otice that eve for such highly structured class of graphs, either the maximum-degree or the Metropolis-Hastigs heuristics discussed i [BDX04] give the optimal solutio Throughout, we use α to deote the commo edge weight of the fastest mixig chai ad µ to deote the optimal SLEM 31 FMMC o edge-trasitive graphs Theorem 31 Suppose the graph G is edge-trasitive, ad let α be the trasitio probability assiged o all the edges The the optimal solutio of the FMMC problem is α { } 1 2 = mi, ν max λ 1 (L) + λ 1 (L) (6) µ { = max 1 λ 1(L), λ } 1(L) λ 1 (L), ν max λ 1 (L) + λ 1 (L) (7) where ν max = max i V ν i is the maximum valecy of the vertices i the graph, ad L is the Laplacia matrix defied i 23 Proof By defiitio of a edge-trasitive graph, there is a sigle orbit of edges uder the actios of its automorphism group Therefore we ca assig the same trasitio probability α o all the edges i the graph (Corollary 23), ad the parametrizatio (3) becomes P = I αl So we have λ i (P) = 1 αλ +1 i (L), i = 1,, ad the SLEM µ(p) = max{λ 2 (P), λ (P)} = max{1 αλ 1 (L), αλ 1 (L) 1} 8

9 Figure 3: The cycle graph C with = 9 To miimize µ(p), we let 1 αλ 1 (L) = αλ 1 (L) 1 ad get α = 2/(λ 1 (L) + λ 1 (L)) But the oegativity costrait P 0 requires that the trasitio probability must also satisfy 0 < α 1/ν max Combiig these two coditios gives the optimal solutio (6) ad (7) We give two examples of FMMC o edge-trasitive graphs 311 Cycles The first example is the cycle graph C ; see Figure 3 The Laplacia matrix is L = which has eigevalues The two extreme eigevalues are 2 2 cos 2kπ, k = 1,, λ 1 (L) = 2 2 cos 2 /2 π, λ 1 (L) = 2 2 cos 2π where /2 deotes the largest iteger that is o larger tha /2, which is /2 for eve or ( 1)/2 for odd By Theorem 31, the optimal solutio to the FMMC problem is α = 2 cos 2π 1 cos 2 /2 π (8) µ = cos 2π 2 /2 π cos 2 cos 2π 2 /2 π cos (9) Whe, the trasitio probability α 1/2 ad the SLEM µ 1 2π 2 / 2 9

10 u v x y Figure 4: The complete bipartite graph K m, with m = 3 ad = Complete bipartite graphs The complete bipartite graph, deoted K m,, has two subsets of vertices with cardialities m ad respectively Each vertex i a subset is coected to all the vertices i the other subset, ad is ot coected to ay of the vertices i its ow subset; see Figure 4 Without loss of geerality, assume m So the maximum degree is ν max = This graph is edge-trasitive but ot vertex-trasitive The Laplacia matrix of this graph is [ ] I L = m 1 m 1 m mi where I m deotes the m by m idetity matrix, ad 1 m deotes the m by matrix whose compoets are all oes For m 2, this matrix has four distict eigevalues m+,, m ad 0, with multiplicities 1, m 1, 1 ad 1, respectively (see 531) By Theorem 31, the optimal trasitio probability o the edges ad the correspodig SLEM are { } 1 α = mi, 2 (10) + 2m { m µ = max, + 2m } (11) 32 Cartesia product of graphs May graphs we cosider ca be costructed by takig Cartesia product of simpler graphs The Cartesia product of two graphs G 1 = (V 1, E 1 ) ad G 2 = (V 2, E 2 ) is a graph with vertex set V 1 V 2, where two vertices (u 1, u 2 ) ad (v 1, v 2 ) are coected by a edge if ad oly if u 1 = v 1 ad {u 2, v 2 } E 2, or u 2 = v 2 ad {u 1, v 1 } E 1 Let G 1 G 2 deote this Cartesia product Its Laplacia matrix is give by L G1 G 2 = L G1 I V1 + I V2 L G2 (12) where deotes the matrix Kroecker product ([Gra81]) The eigevalues of L G1 G 2 are give by λ i (L G1 ) + λ j (L G2 ), i = 1,, V 1, j = 1,, V 2 (13) where each eigevalue is obtaied as may times as its multiplicity (eg, [Moh97]) The adjacecy matrix of the Cartesia product of graphs also has similar properties, which we will use later for distace-trasitive graphs Detailed backgroud o spectral graph theory ca be foud i, eg, [Big74, DCS80, Chu97, GR01] Combiig Theorem 31 ad the above expressio for eigevalues, we ca easily obtai solutios to the FMMC problem o graphs formed by takig Cartesia product of simple graphs 10

11 Figure 5: The two-dimesioal mesh with wraparouds M with = Two-dimesioal meshes Here we cosider the two-dimesioal mesh with wraparouds at two eds of each row ad colum, see Figure 5 It is the Cartesia product of two copies of C We write it as M = C C By equatio (13), its Laplacia matrix has eigevalues 4 2 cos 2iπ 2jπ 2 cos, i, j = 1,, By Theorem 31, we obtai the optimal trasitio probability α 1 = 3 2 cos 2 /2 π cos 2π ad the smallest SLEM 2 /2 π µ 1 2 cos + cos 2π = 3 2 cos 2 /2 π cos 2π Whe, the trasitio probability α 1/4 ad the SLEM µ 1 π 2 / Hypercubes The d-dimesioal hypercube, deoted Q d, has 2 d vertices, each labeled with a biary word with legth d Two vertices are coected by a edge if their words differ i exactly oe compoet (see Figure 6) This graph is isomorphic to the Cartesia product of d copies of K 2, the complete graph with two vertices The Laplacia of K 2 is [ L K2 = whose two eigevalues are 0 ad 2 The oe-dimesioal hypercube Q 1 is just K 2 Higher dimesioal hypercubes are defied recursively: By equatio (12), their Laplacia matrices are ], Q k+1 = Q k K 2, k = 1, 2, L Qk+1 = L Qk I 2 + I 2 k L K2, k = 1, 2, Usig equatio ( (13) recursively, the Laplacia of Q d has eigevalues 2k, k = 0, 1,,d, each with multiplicity d k) The FMMC is achieved for: α = 1 d + 1, µ = d 1 d + 1 This solutio has also bee worked out, for example, i [Moh97] 11

12 Figure 6: The hypercubes Q 1, Q 2 ad Q 3 33 FMMC o distace-trasitive graphs Distace-trasitive graphs have bee studied extesively i the literature (see, eg, [BCN89]) I particular, they are both edge- ad vertex-trasitive I previous examples, the cycles ad the hypercubes are actually distace-trasitive graphs; so are the bipartite graphs whe the two parties have equal umber of vertices I a distace-trasitive graph, all vertices have the same valecy, which we deote by ν The Laplacia matrix ca be writte as L = νi A, with A beig the adjacecy matrix Therefore λ i (L) = ν λ +1 i (A), i = 1,, We ca substitute the above equatio i equatios (6) ad (7) to obtai the optimal solutio i terms of λ 2 (A) ad λ (A) Sice distace-trasitive graphs usually have very large automorphism groups, the eigevalues of the adjacecy matrix A (ad the Laplacia L) ofte have very high multiplicities But to solve the FMMC problem, we oly eed to kow the distict eigevalues; actually, oly λ 2 (A) ad λ (A) would suffice I this regard, it is more coveiet to use a much smaller matrix, the itersectio matrix, which has all the distict eigevalues of the adjacecy matrix Let D be the diameter of the graph For a oegative iteger k D, choose ay two vertices u ad v such that their distace satisfies δ(u, v) = k Let a k, b k ad c k be the umber of vertices that are adjacet to u ad whose distace from v are k, k + 1 ad k 1, respectively That is, a k = {w V δ(u, w) = 1, δ(w, v) = k} b k = {w V δ(u, w) = 1, δ(w, v) = k + 1} c k = {w V δ(u, w) = 1, δ(w, v) = k 1} For distace-trasitive graphs, these umbers are idepedet of the particular pair of vertices u ad v chose Clearly, we have a 0 = 0, b 0 = ν ad c 1 = 1 The itersectio matrix B is the followig tridiagoal (D + 1) (D + 1) matrix B = a 0 b 0 c 1 a 1 b 1 c 2 a 2 bd 1 c D a D Deote the eigevalues of the itersectio matrix, i decreasig order, as η 0, η 1,, η D These are precisely the (D + 1) distict eigevalues of the adjacecy matrix A (see, eg, [Big74]) I particular, we have λ 1 (A) = η 0 = ν, λ 2 (A) = η 1, λ (A) = η D 12

13 The followig corollary is a direct cosequece of Theorem 31 Corollary 32 The optimal solutio of the FMMC problem o a distace-trasitive graph is { } 1 α = mi ν, 2 (14) 2ν (η 1 + η D ) { } µ η1 = max ν, η 1 η D (15) 2ν (η 1 + η D ) Next we give solutios for the FMMC problem o several families of distace-trasitive graphs 331 Complete graphs The case of the complete graph with vertices, usually called K, is very simple It is distacetrasitive, with diameter D = 1 ad valecy ν = 1 The itersectio matrix is [ ] 0 1 B =, 1 2 with eigevalues η 0 = 1, η 1 = 1 Usig equatios (14) ad (15), the optimal parameters are α = 1, µ = 0 The associated matrix P = (1/)11 T has oe eigevalue equal to 1, ad the remaiig 1 eigevalues vaish Such Markov chais achieve perfect mixig after just oe step, regardless of the value of 332 Peterse graph The Peterse graph, show i Figure 7, is a well-kow distace-trasitive graph with 10 vertices ad 15 edges The diameter of the graph is D = 2, ad the itersectio matrix is B = with eigevalues η 0 = 3, η 1 = 1 ad η 2 = 2 Applyig the formula (14) ad (15), we obtai α = 2 7, µ = Hammig graphs The Hammig graphs, deoted H(d, ), have vertices labeled by elemets i the Cartesia product {1,,} d, with two vertices beig adjacet if they differ i exactly oe compoet By the defiitio, it is clear that Hammig graphs are isomorphic to the Cartesia product of d copies of the complete graph K Hammig graphs are distace-trasitive, with diameter D = d ad valecy ν = d ( 1) Their eigevalues are give by η k = d ( 1) k for k = 0,,d These 13

14 Figure 7: The Peterse graph ca be obtaied usig a equatio for eigevalues of adjacecy matrices, similar to (13), with the eigevalues of K beig 1 ad 1 Therefore the FMMC has parameters: { } α 1 = mi d ( 1), 2 (d + 1) { µ = max 1 d( 1), d 1 } d + 1 We ote that hypercubes (see 322) are special Hammig graphs with = Johso graphs The Johso graph J(, q) (for 1 q /2) is defied as follows: the vertices are the q-elemet subsets of {1,,}, with two vertices beig coected with a edge if ad oly if the subsets differ exactly by oe elemet It is a distace-trasitive graph, with ( ) q vertices ad 1 2 q ( q)( ) q edges It has valecy ν = q ( q) ad diameter D = q The eigevalues of the itersectio matrix ca be computed aalytically ad they are: η k = q ( q) + k (k 1), k = 0,,q Therefore, by Corollary 32, we obtai the optimal trasitio probability { } α 1 = mi q ( q), 2 q + + q q 2 ad the smallest SLEM { µ = max 1 4 FMMC o orbit graphs q( q), 1 } 2 q + + q q 2 For graphs with large automorphism groups, the eigevalues of the trasitio probability matrix ofte have very high multiplicities To solve the FMMC problem, it suffices to work with oly the distict eigevalues without cosideratio of their multiplicities This is exactly what the itersectio matrix does for distace-trasitive graphs I this sectio we develop similar tools for more geeral graphs More specifically, we show how to costruct a orbit chai which is much smaller i size tha the origial Markov chai, but cotais all its distict eigevalues (with much fewer multiplicities) The FMMC o the origial graph ca be foud by solvig a much smaller problem o the orbit chai 14

15 41 Orbit theory Here we review the orbit theory developed i [BDPX05] Let P be a symmetric Markov chai o the graph G = (V, E), ad H be a group of automorphisms of the graph Ofte, it is a subgroup of the full automorphism group Aut(G) The vertex set V partitios ito orbits O v = {hv : h H} For otatioal coveiece, i this sectio we use P(v, u), for v, u V, to deote etries of the trasitio probability matrix We defie the orbit chai by specifyig the trasitio probabilities betwee orbits P H (O v, O u ) = P(v, O u ) = u O u P(v, u ) (16) This trasitio probability is idepedet of which v O(v) is chose, so it is well defied ad the lumped orbit chai is ideed Markov The orbit chai is i geeral o loger symmetric, but it is always reversible Let π(i), i V, be the statioary distributio of the origial Markov chai The the statioary distributio o the orbit chai is obtaied as π H (O v ) = i O v π(i) (17) It ca be verified that π H (O v )P H (O v, O u ) = π H (O u )P H (O u, O v ), (18) which is the detailed balace coditio to test reversibility The followig is a summary of the orbit theory we developed i [BDPX05], which relate the eigevalues ad eigevectors of the orbit chai P H to the eigevalues ad eigevectors of the origial chai P Liftig ([BDPX05, 31]) If λ is a eigevalue of P H with associated eigevector f, the λ is a eigevalue of P with H-ivariat eigefuctio f(v) = f(o v ) Coversely, every H-ivariat eigefuctio appears uiquely from this costructio Projectio ([BDPX05, 32]) Let λ be a eigevalue of P with eigevector f Defie f(o v ) = h H f(h 1 (v)) The λ appears as a eigevalue of P H, with eigevector f, if either of the followig two coditios holds: (a) H has a fixed poit v ad f(v ) 0 (b) f is ozero at a vertex v i a Aut(G)-orbit which cotais a fixed poit of H Equipped with this orbit theory, we would like to costruct oe or multiple orbit chais that retai all the eigevalues of the origial chai Ideally the orbit chais are much smaller i size tha the origial chai, with the eigevalues havig much fewer multiplicities The followig theorem (Theorem 37 i [BDPX05]) gives sufficiet coditios that guaratee that the orbit chai(s) attai all the eigevalues of the origial chai Theorem 41 Suppose that V = O 1 O K is a disjoit uio of the orbits uder Aut(G) Let H i be the subgroup of Aut(G) that has a fixed poit i O i The all eigevalues of P occur amog the eigevalues of {P Hi } K i=1 Further, every eigevector of P occurs by liftig a eigevector of some P Hi Observe that if H G Aut(G), the the eigevalues of P H cotai all eigevalues of P G This allows disregardig some of the H i i Theorem 41 I particular, it is possible to costruct a sigle orbit chai that cotais all eigevalues of the origial chai Therefore we have 15

16 p O u p mp O v (a) Orbit chai uder S m S ( 1)p O u mp mp p O v y (c) Orbit chai uder S m S 1 O u (m 1)p p p O v x (b) Orbit chai uder S m 1 S ( 1)p O u (m 1)p O v (m 1)p ( 1)p p p x p y (d) Orbit chai uder S m 1 S 1 Figure 8: Orbit chais of K m, uder differet automorphism groups The vertices labeled O u ad O v are orbits of vertices u ad v (labeled i Figure 4) uder correspodig actios The vertices labeled x ad y are fixed poits Corollary 42 Suppose that V = O 1 O k is a disjoit uio of the orbits uder Aut(G), ad H is a subgroup of Aut(G) If H has a fixed poit i every O i, the all distict eigevalues of P occur amog the eigevalues of P H Remarks To fid H i the above corollary, we ca just compute the correspodig stabilizer, ie, compute the largest subgroup of Aut(G) that fixes oe poit i each orbit Note that the H promised by the corollary may be trivial i some cases; see the example i 536 We illustrate the orbit theory with the bipartite graph K m, show i Figure 4 It is easy to see that Aut(K m, ) is the direct product of two symmetric groups, amely S m S, with each symmetric group permutig oe of the two subsets of vertices This graph is edge-trasitive So we assig the same trasitio probability p o all the edges The orbit chais uder four differet subgroups of Aut(K m, ) are show i Figure 8 The trasitio probabilities betwee orbits are calculated usig equatio (16) Sice the trasitio probabilities are ot symmetric, we represet the orbit chais by directed graphs, with differet trasitio probabilities labeled o opposite directios betwee two adjacet vertices The full automorphism group Aut(K m, ) has two orbits of vertices; see Figure 8(a) The orbit graphs uder the subgroups S m 1 S (Figure 8(b)) ad S m S 1 (Figure 8(c)) each cotais a fixed poit of the two orbits uder Aut(K m, ) By Theorem 41, these two orbit chais cotai all the distict eigevalues of the origial chai o K m, Alteratively, we ca costruct the orbit chai uder the subgroup S m 1 S 1, show i Figure 8(d) This orbit chai cotai a fixed poit i both orbits uder Aut(K m, ) By Corollary 41, all distict eigevalues of K m, appear i this orbit chai I particular, this shows that there are at most four distict eigevalues i the origial chai 16

17 If we order the vertices i Figure 8(d) as (x, y, O u, O v ), the the trasitio probability matrix for this orbit chai is 1 p p 0 ( 1)p P H = p 1 mp (m 1)p 0 0 p 1 p ( 1)p p 0 (m 1)p 1 mp where H = S m 1 S 1 By equatio (17), its statioary distributio is ( ) 1 π H = m +, 1 m +, m 1 m +, 1 m + 42 Fastest mixig reversible Markov chai o orbit graph Sice i geeral the orbit chai is o loger symmetric, we caot directly use the covex optimizatio formulatio (1) or (2) to miimize µ(p H ) Fortuately, the detailed balace coditio (18) leads to a simple trasformatio that allow us to formulate the problem of fidig the fastest reversible Markov chai as a covex program [BDX04] Suppose the orbit chai P H cotais all distict eigevalues of the origial chai Let π H be the statioary distributio of the orbits, ad let Π = Diag(π H ) The detailed balace coditio (18) ca be writte as ΠP H = P T H Π, which implies that the matrix Π1/2 P H Π 1/2 is symmetric (ad of course, has the same eigevalues as P H ) The eigevector of Π 1/2 P H Π 1/2 associated with the maximum eigevalue 1 is q = ( π H (O 1 ),, π H (O k )) The SLEM µ(p H ) equals the spectral orm of Π 1/2 P H Π 1/2 restricted to the orthogoal complemet of the subspace spaed by q This ca be writte as µ(p H ) = (I qq T )Π 1/2 P H Π 1/2 (I qq T ) 2 = Π 1/2 P H Π 1/2 qq T 2 Itroducig a scalar variable s to boud the above spectral orm, we ca formulate the fastest mixig reversible Markov chai problem as a SDP miimize s subject to si Π 1/2 P H Π 1/2 qq T si P H 0, P H 1 = 1, ΠP H = PH TΠ P H (O, O ) = 0, (O, O ) / E H (19) The optimizatio variables are the matrix P H ad scalar s, ad problem data is give by the orbit graph ad the statioary distributio π H Note that the reversibility costrait ΠP H = PH TΠ ca be dropped sice it is always satisfied by the costructio of the orbit chai; see equatio (18) By pre- ad post-multiplyig the matrix iequality by Π 1/2, we ca write the aother equivalet formulatio: miimize s subject to sπ ΠP H π H π T H sπ P H 0, P H 1 = 1, P H (O, O ) = 0, (O, O ) / E H To solve the fastest mixig reversible Markov chai problem o the orbit graph, we eed the followig three steps (20) 17

18 1 Coduct symmetry aalysis o the origial graph: idetify the automorphism graph Aut(G) ad determie the umber of orbits of edges N By Corollary 22, this is the umber of trasitio probabilities we eed to cosider 2 Fid a group of automorphisms H that satisfies the coditios i Corollary 42 Costruct its orbit chai by computig the trasitio probabilities usig equatio (16), ad compute the statioary distributio usig equatio (17) Note that the etries of P H are multiples of the trasitio probabilities o the origial graph 3 Solve the fastest mixig reversible Markov chai problem (19) The optimal SLEM µ(p H ) is also the optimal SLEM for the origial chai, ad the optimal trasitio probabilities o the origial chai ca be obtaied by simple scalig of the optimal orbit trasitio probabilities We have assumed a sigle orbit chai that cotais all distict eigevalues of the origial chai Sometimes it is more coveiet to use multiple orbit chais Let P Hi, i = 1,,K, be the collectio of orbit chais i Theorem 41 I this case we eed to miimize max i µ(p Hi ) This ca be doe by simply addig the set of costraits i (19) for every matrix P Hi For example, for the complete bipartite graph K m,, istead of usig the sigle orbit chai i Figure 8(d), we ca use the two orbit chais i Figure 8(b) ad Figure 8(c) together, with two sets of costraits i the SDP (19) 43 Examples We demostrate the above computatioal procedure o orbit graphs with two examples: the graph K -K ad the complete biary tree Both examples will be revisited i 5 usig the method of block diagoalizatio 431 The graph K -K The graph K -K cosists of two copies of the complete graph K joied by a bridge (see Figure 9(a)) We follow the three steps described i 42 First, it is clear by ispectio that the full automorphism group is C 2 (S 1 S 1 ) The actios of S 1 S 1 are all possible permutatios of the two set of 1 vertices, distict from the two ceter vertices x ad y, amog themselves The group C 2 acts o the graph by switchig the two halves The semi-direct product symbol meas that the actios of S 1 S 1 ad C 2 do ot commute By symmetry aalysis i 2, there are three edge orbits uder the full automorphism group: the bridgig edge betwee vertices x ad y, the edges coectig x ad y to all other vertices, ad the edges coectig all other vertices Thus it suffices to cosider just three trasitio probabilities p 0, p 1, ad p 2, each labeled i Figure 9(a) o oe represetative of the three edge orbits As the secod step, we costruct the orbit chais The orbit chai of K -K uder the full automorphism group is depicted i Figure 9(b) The orbit O x icludes vertices x ad y, ad the orbit O z cosists of all other 2( 1) vertices The trasitio probabilities of this orbit chai are calculated from equatio (16) ad are labeled o the directed edges i Figure 9(b) Similarly, the orbit chai uder the subgroup S 1 S 1 is depicted i Figure 9(c) While these two orbit chais are the most obvious to costruct, oe of them cotais all eigevalues of the origial chai, or does their combiatio For the oe i Figure 9(b), the full automorphism group does ot have a fixed poit either of its orbit O x or O z For the oe i 9(c), the automorphism group S 1 S 1 has a fixed poit i O x (either x or y), but does ot have a fixed poit i O z (ote here O z is the 18

19 p 2 z p 1 x p 0 y u v (a) The graph K -K ( 1)p 1 O z O x p 1 (b) Orbit chai uder C 2 (S 1 S 1 ) O u ( 1)p 1 p 0 ( 1)p 1 x y p 1 p 1 O v (c) Orbit chai uder S 1 S 1 z ( 2)p 2 O u p 1 p 2 p 1 p p 0 1 x ( 2)p 1 ( 1)p 1 y O v (d) Orbit chai uder S 2 S 1 Figure 9: The graph K -K ad its orbit chais uder differet automorphism groups Here O x, O z, O u, O v represet orbits of the vertices x, z, u, v (labeled i Figure 9(a)), respectively, uder the correspodig automorphism groups i each subgraph 19

20 orbit of z uder the full automorphism group) To fix the problem, we cosider the orbit chai uder the group S 2 S 1, which leave the vertex x, y, ad z fixed, while permutig the rest 2 vertices o the left ad the 1 poits o the right, respectively The correspodig orbit chai is show i Figure 9(d) By Corollary 42, all distict eigevalues of the origial Markov chai o K -K appear as eigevalues of this orbit chai Thus there are at most five distict eigevalues i the origial chai o matter how large is To fiish the secod step, we calculate the trasitio probabilities of the orbit chai uder H = S 2 S 1 usig equatio (16) ad label them i Figure 9(d) If we order the vertices of this orbit chai as (x, y, z, O u, O v ), the the trasitio probability matrix o the orbit chai is P H = 1 p 0 ( 1)p 1 p 0 p 1 ( 2)p 1 0 p 0 1 p 0 ( 1)p ( 1)p 1 p p 1 ( 2)p 2 ( 2)p 2 0 p 1 0 p 2 1 p 1 p p p 1 By equatio (17), the statioary distributio of the orbit chai is ( ) 1 π H = 2, 1 2, 1 2, 2 2, 1 2 As the third step, we solve the SDP (19) with the above parametrizatio It is remarkable to see that we oly eed to solve a SDP with 4 variables (three trasitio probabilities p 0, p 1, p 2, ad the extra scalar s) ad 5 5 matrices o matter how large the graph (the umber ) is We will revisit this example i 534 usig the block diagoalizatio method, where we preset a aalytic expressio for the exact optimal SLEM ad correspodig trasitio probabilities 432 Complete biary tree We cosider a complete biary tree with levels of braches, deoted as T The total umber of odes is V = The matrix iequalities i the correspodig SDP have size V V, which is clearly expoetial i However, the biary tree has a very large automorphism group, of size 2 (2 1) This automorphism group is best described recursively Plaily, for = 1, we have Aut(T 1 ) = S 2 For > 1, it ca be obtaied by the recursio Aut(T k+1 ) = Aut(T k ) S 2, k = 1,, 1, where represets the wreath product of two groups (eg, [JK81]) More specifically, let g = (g 1, g 2 ) ad h = (h 1, h 2 ) be elemets of the product group Aut(T k ) Aut(T k ), ad σ ad π be i S 2 The multiplicatio rule of the wreath product is (g, σ)(h, π) = ( (g 1 h σ 1 (1), g 2 h σ 1 (2)), σπ ) This is a semi-direct product Aut(T k ) 2 S 2 (cf the automorphism group of K -K ) From the above recursio, the automorphism group of T is Aut(T ) = S 2 S 2 S 2 ( times) (The wreath product is associative, but ot commutative) The represetatio theory of the automorphism group of the biary tree has bee thoroughly studied as this group is the Sylow 2-subgroup of a symmetric group; see [OOR04, AV05] 20

21 2p 1 p 1 2p 2 p 2 2p 3 p 3 (a) Orbit graph ad chai uder S 2 S 2 S 2 (b) Orbit graph uder (S 2 S 2 ) (S 2 S 2 ) (c) Orbit graph uder (S 2 S 2 ) (S 2 S 2 ) (d) Orbit graph uder S 2 (S 2 S 2 ) Figure 10: Orbit graphs of the complete biary tree T ( = 3) uder differet automorphism groups The vertices surrouded by a circle are fixed poits of the correspodig automorphism group The orbit graph of T uder its full automorphism group is a path with +1 odes (Figure 10(a), left) Sice there are orbits of edges, there are differet trasitio probabilities we eed to cosider We label them as p k, k = 1,,, from top to bottom of the tree The correspodig orbit chai, represeted by a directed graph labeled with trasitio probabilities betwee orbits, is show o the right of Figure 10(a) To simplify presetatio, oly the orbit graphs are show i other subfigures of Figure 10 The correspodig orbit chais should be straightforward to costruct The largest subgroup of Aut(T ) that has a fixed poit i every orbit uder Aut(T ) is 1 W = (S 2 S 2 ) (k times) k=1 where deotes direct product of groups The correspodig orbit graph is show i Figure 10(d) for = 3 The umber of vertices i this orbit graph is ( ) ( + 1) = = 1 ( + 1)( + 2), 2 2 which is much smaller tha , the size of T From the above aalysis, we oly eed to solve the fastest reversible Markov chai problem o the orbit graph of size ( ) +1 2 with variables p1,,p I ext sectio, usig the techique of block diagoalizatio, we will see that the trasitio probability matrix of size ( ) +1 2 ca be further decomposed ito smaller matrices with sizes 1, 2,,+1 Due to a eigevalue iterlacig result, we oly eed to cosider the orbit chai with vertices i Figure 10(b) 21

22 5 Symmetry reductio by block diagoalizatio By defiitio of the fixed-poit subspace F (i 22), ay trasitio probability matrix P F is ivariat uder the actios of Aut(G) More specifically, for ay permutatio matrix Q give by σ Aut(G), we have QPQ T = P, equivaletly QP = PQ I this sectio we show that this property allows the costructio of a coordiate trasformatio matrix that ca block diagoalize every P F The resultig blocks usually have much smaller sizes ad repeated blocks ca be discarded i computatio The method we use i this sectio is based o classical group represetatio theory (eg, [Ser77]) It was developed for more geeral SDPs i [GP04], ad has foud applicatios i sumof-squares decompositio for miimizig polyomial fuctios [Par00, Par03, PS03] ad cotroller desig for symmetric dyamical systems [CLP03] 51 Some group represetatio theory Let G be a group A represetatio ρ of G assigs a ivertible matrix ρ(g) to each g G i such a way that the matrix assiged to the product of two elemets i G is the product of the matrices assiged to each elemet: ρ(gh) = ρ(g)ρ(h) The matrices we work with are all ivertible ad are cosidered over the real or complex umbers We thus regard ρ as a homomorphism from g to the liear maps o a vector space V The dimesio of ρ is the dimesio of V Two represetatios are equivalet if they are related by a fixed similarity trasformatio If W is a subspace of V ivariat uder G, the ρ restricted to W gives a subrepresetatio Of course the zero subspace ad the subspace W = V are trivial subrepresetatios If the represetatio ρ admits o o-trivial subrepresetatio, the ρ is called irreducible We cosider first complex represetatios, as the theory is cosiderably simpler i this case For a fiite group G there are oly fiitely may iequivalet irreducible represetatios ϑ 1,,ϑ h of dimesios 1,, h, respectively The degrees i divide the group order G, ad satisfy the coditio h i=1 2 i = G Every liear represetatio of G has a caoical decompositio as a direct sum of irreducible represetatios ρ = m 1 ϑ 1 m 2 ϑ 2 m h ϑ h, where m 1,,m h are the multiplicities Accordigly, the represetatio space C has a isotypic decompositio C = V 1 V h (21) where each isotypic compoets cosists of m i ivariat subspaces V i = V 1 i V m i i, (22) each of which has dimesio i ad trasforms after the maer of ϑ i A basis of this decompositio trasformig with respect to the matrices ϑ i (g) is called symmetry-adapted ad ca be computed usig the algorithm preseted i [Ser77, 26-27] or [FS92, 52] This basis defies a chage of coordiates by a matrix T collectig the basis as colums By Schur s lemma, if a matrix P satisfies ρ(g)p = Pρ(g), g G, (23) the T 1 PT has block diagoal form with oe block P i for each isotypic compoet of dimesio m i i, which further decomposes ito i equal blocks B i of dimesio m i That is P 1 0 B i 0 T 1 PT =, P i = (24) 0 P h 0 B i 22

23 For our applicatio of semidefiite programs, the problems are preseted i terms of real matrices, ad therefore we would like to use real coordiate trasformatios I fact a geeralizatio of the classical theory to the real case is preseted i [Ser77, 132] If all ϑ i (g) are real matrices the irreducible represetatio is called absolutely irreducible Otherwise, for each ϑ i with complex character its complex cojugate will also appear i the caoical decompositio Sice ρ is real both will have the same multiplicity ad real bases of V i + V i ca be costructed So two complex cojugate irreducible represetatios form oe real irreducible represetatio of complex type There is a third case, real irreducible represetatios of quateroia type, rarely see i practical examples I this paper, we assume that the represetatio ρ is orthogoal, ie, ρ(g) T ρ(g) = ρ(g)ρ(g) T = I for all g G As a result, the trasformatio matrix T ca also be chose to be orthogoal Thus T 1 = T T (for complex matrices, it is the cojugate traspose) For symmetric matrices the block correspodig to a represetatio of complex type or quateroia type simplifies to a collectio of equal subblocks For the special case of circulat matrices, complete diagoalizatio reveals all the eigevalues [Dia88, page 50] 52 Block diagoalizatio of SDP costrait As i 22, for every σ Aut(G) we assig a permutatio matrix Q(σ) by lettig Q ij (σ) = 1 if i = σ(j) ad Q ij (σ) = 0 otherwise This is a -dimesioal represetatio of Aut(G), which is ofte called the atural represetatio As metioed i the begiig of this sectio, every matrix P i the fixed-poit subset F has the symmetry of Aut(G); ie, it satisfies the coditio (23) with ρ = Q Thus a coordiate trasformatio matrix T ca be costructed such that P ca be block diagoalized ito the form (24) Now we cosider the SDP (5), which is the FMMC problem formulated i the fixed-poit subset F I 23, we have derived the expressio P(p) = I N k=1 p kl k, where L k is the Laplacia matrix for the kth orbit graph ad p k is the commo trasitio probability assiged o all edges i the kth orbit graph Note the matrix P(p) has the symmetry of Aut(G) Applyig the coordiate trasformatio T to the liear matrix iequalities, we obtai the followig equivalet problem miimize s subject to si mi B i (p) J i si mi, i = 1,,h p k 0, k = 1,,N N k=1 (L k) ii p k 1, i = 1,, (25) where B i (p) correspod to the small blocks B i i (24) of the trasformed matrix T T P(p)T, ad J i are the correspodig diagoal blocks of T T (1/)11 T T The umber of matrix iequalities h is the umber of iequivalet irreducible represetatios, ad the size of each matrix iequality m i is the multiplicity of the correspodig irreducible represetatio Note that we oly eed oe out of i copies of each B i i the decompositio (24) Sice m i ca be much smaller tha (the umber of vertices i the graph), the improvemet i computatioal complexity over the SDP formulatio (5) ca be sigificat (see the flop couts discussed i 12) This is especially the case whe there are high-dimesioal irreducible represetatios (ie, whe i is large; see, eg, K -K defied i 431) The trasformed SDP formulatio (25) eeds some further justificatio Namely, all the offdiagoal blocks of the matrix T T (1/)11 T T have to be zero This is i fact the case Moreover, the followig theorem reveals a iterestig coectio betwee the block diagoalizatio approach ad the orbit theory i 4 23