arxiv: v1 [stat.ml] 26 Sep 2012

Transcription

1 Reversible MCMC on Markov equivalence classes of sparse direced acyclic graphs arxiv: v1 [sa.ml] 26 Sep 2012 Yangbo He 1, Jinzhu Jia 2 and Bin Yu 3 1 School of Mahemaical Sciences and Cener of Saisical Science, LMAM, LMEQF Peking Universiy heyb@mah.pku.edu.cn 2 School of Mahemaical Sciences and Cener of Saisical Science, LMAM Peking Universiy jzjia@mah.pku.edu.cn 3 Deparmen of Saisics UC, Berkeley,CA binyu@sa.berkeley.edu Absrac: Graphical models are popular saisical ools which are used o represen dependen or causal complex sysems. Saisically equivalen causal or direced graphical models are said o belong o a Markov equivalen class. I is of grea ineres o describe and undersand he space of such classes. However, wih currenly known algorihms, sampling over such classes is only feasible for graphs wih fewer han approximaely 20 verices. In his paper, we design reversible irreducible Markov chains on he space of Markov equivalen classes by proposing a perfec se of operaors ha deermine he ransiions of he Markov chain. The saionary disribuion of a proposed Markov chain has a closed form and can be compued easily. Especially, we consruc a concree perfec se of operaors on sparse Markov equivalence classes by inroducing appropriae condiions on each possible operaor. Algorihms and heir acceleraed version are provided o efficienly generae Markov chains and o explore properies of Markov equivalence classes of sparse direced acyclic graphs (DAGs) wih housands of verices. We find experimenally ha in mos Markov equivalence classes of sparse DAGs, (1) mos edges are direced, (2) mos undireced subgraphs are small, and (3) he number of hese undireced subgraphs grows approximaely linearly wih he number of verices. Keywords and phrases: Sparse graphical model, Reversible Markov chain, Markov equivalence class. This work is parly done when Yangbo He was visiing Deparmen of Saisics in UC Berkeley. Jinzhu Jia s work was done when he was a posdoc in UC Berkeley. We are very graeful o Adam Bloniarz for his commens ha significanly improved he presenaion of our manuscrip. We also hank Jasjee Sekhon and Lan Wu for heir helpful commens. Suppored parially by NSFC ( , ), 973 Program-2007CB814905, and LMEQF. Suppored parially by NSFC ,DPHEC , US NSF gran SES (CDI). Suppored parially by US NSF grans DMS , DMS , DMS , SES (CDI), ARO gran W911NF , and he Cener for Science of Informaion (CSoI), an US NSF Science and Technology Cener, under gran agreemen CCF

2 1. Inroducion Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs 2 Graphical models based on direced acyclic graphs ( DAGs, denoed as D) are widely used o represen causal or dependen relaionships in various scienific invesigaions, such as bioinformaics, epidemiology, sociology and business [11, 12, 17, 18, 22, 27, 31]. A DAG encodes he independence and condiional independence resricions (Markov properies) of variables. However, because differen DAGs can encode he same se of independencies or condiional independencies, mos of he ime we canno disinguish DAGs via observaional daa [28]. A Markov equivalence class is used o represen all DAGs ha encode he same dependencies and independencies [2, 6, 29]. A Markov equivalence class can be visualized (or modeled) and uniquely represened by a compleed parial direced acyclic graph (compleed PDAG for shor) [6] which possibly conains boh direced edges and undireced edges [20]. There exiss a one-o-one correspondence beween compleed PDAGs and Markov equivalence classes [2]. The compleed PDAGs are also called essenial graphs by Andersson e al. [2] and maximally oriened graphs by Meek [24]. A se of compleed PDAGs can be used as a model space. The modeling ask is o discover a proper Markov equivalence class in he model space [3, 4, 7, 8, 16, 23]. Undersanding he se of Markov equivalence classes is imporan and useful for saisical causal modeling [13, 14, 19]. For example, if he number of DAGs is large for Markov equivalence classes in he model space, searching based on unique compleed PDAGs could be subsanially more efficien han searching based on DAGs [6, 23, 25]. Moreover, if mos compleed PDAGs in he model space have many undireced edges (wih non-idenifiable direcions), many inervenions migh be needed o idenify he causal direcions [10, 15]. Because he number of Markov equivalence classes increases super-exponenially wih he number of verices (eg. more han classes wih 10 verices)[14], i s hard o sudy ses of Markov equivalence classes. To our knowledge, only whole se ha conains all compleed PDAGs wih a small given number of verices ( 10) have been sudied horoughly in he lieraure [13, 14, 30]. Moreover, hese sudies focus on he size of Markov equivalence classes, which is defined as he number of DAGs in a Markov equivalence class. Gillispie and Perlman [14] obain he rue size disribuion of all Markov equivalence classes wih a given number (10 or fewer) of verices by lising all classes. Pena [30] designs a Markov chain o esimae he proporion of he equivalence classes conaining only one DAG for graphs wih 20 or fewer verices. In recen years, sparse graphical models have become popular ools for fiing highdimensional mulivariae daa. The sparsiy assumpion inroduces resricions on he model space; a sandard resricion is ha he number of edges in he graph be less han a small muliple of he number of verices. I is hus boh ineresing and imporan o be able o explore he properies of subses of sparse graphical models. In his paper, we inroduce a reversible irreducible Markov chain on general Markov equivalence classes. This Markov chain allows one o sudy properies of he ses ha conain sparse Markov equivalence classes in a compuaionally efficien manner, for sparse graphs wih housands of verices A Markov equivalence class and is represenaion A graph G is defined as a pair (V, E), where V = {x 1,,x p } denoes he verex se wih p variables and E denoes he edge se. Le n G = E be he number of edges in G. A direced

3 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs 3 (undireced) edge is denoed as or ( ). A graph is direced (undireced) if all of is edges are direced (undireced). A sequence (x 1,x 2,,x k ) of disinc verices is called a pah from x 1 o x k if eiher x i x i+1 or x i x i+1 is in E for all i =1,..., k 1. A pah is parially direced if a leas one edge in i is direced. A pah is direced (undireced) if all edges are direced(undireced). A cycle is a pah from a verex o iself. A direced acyclic graph (DAG), denoed by D, is a direced graph which does no conain any direced cycle. Le τ be a subse of V. The subgraph D τ =(τ, E τ ) induced by he subse τ has verex se τ and edge se E τ, he subse of E which conains he edges wih boh verices in τ. A subgraph x z y is called a v-srucure if here is no edge beween x and y. An acyclic parially direced graph (PDAG), denoed by P, is a graph wih no direced cycle. A graphical model consiss of a DAG and a join probabiliy disribuion. Wih he graphical model, all condiional independencies implied by he join probabiliy disribuion can be read from he DAG. These condiional independencies are called Markov properies of he DAG [28]. A Markov equivalence class (MEC) is a se of DAGs ha have he same Markov properies. Le he skeleon of an arbirary graph G be he undireced graph wih he same verices and edges as G, regardless of heir direcions. Verma and Pearl [32] proved he following characerizaion of Markov equivalence classes: Lemma 1 (Verma and Pearl [32]). Two DAGs are Markov equivalen if and only if hey have he same skeleon and he same v-srucures. This lemma implies ha, among DAGs in an equivalence class, some edge orienaions may vary, while ohers will be preserved (for example, hose involved in a V-srucure). Consequenly, a Markov equivalence class can be represened uniquely by a compleed PDAG, defined as follows: Definiion 1 (Compleed PDAG [6]). The compleed PDAG of a DAG D, denoed as C, is a PDAG ha has he same skeleon as D, and an edge is direced in C if and only if i has he same orienaion in every equivalen DAG of D. In oher words, a compleed PDAG of a DAG D has he same skeleon as D and keeps he v-srucures of D. Anoher popular name of a compleed PDAG is essenial graph inroduced by Andersson e al. [2], who show ha all direced edges in a compleed PDAG mus be srongly proeced, defined as follows: Definiion 2. Le G =(V, E) be a graph. A direced edge v u E is srongly proeced in G if v u E occurs in a leas one of he four induced subgraphs of G win 1 Figure 1: (a) :v u (b) :v u (c) :v u (d) :v u, (w w 1 ) w w w w Fig 1. Four configuraions where v u is srongly proeced in G. Andersson e al. [2] also inroduce necessary and sufficien condiions for a graph o be an essenial graph (equivalenly, a compleed PDAG), which we include in Lemma 2 in

4 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs 4 Appendix A. If we delee all direced edges from a compleed PDAG, we are lef wih several isolaed undireced subgraphs. Each isolaed undireced subgraph is a chain componen of he compleed PDAG. Observaional daa is no sufficien o learn he direcions of undireced edges of a compleed PDAG; one mus perform addiional inervenion experimens. In general, he size of a chain componen is a measure of complexiy of causal learning; he larger chain componens he more inervenions necessary o learn he underlying causal graph [15]. In learning graphical models [6] or sudying Markov equivalence classes [30], Markov chains on compleed PDAGs play an imporan role. We briefly inroduce he exising mehods o consruc Markov chains on compleed PDAGs in he nex subsecion Markov chains on compleed PDAGs To consruc a Markov chain on compleed PDAGs, we need o generae he ransiions among hem. In general, an operaor ha can modify he iniial compleed PDAG locally can be used o carry ou a ransiion [6, 25, 30]. Le C be a compleed PDAG. We consider six ypes of operaors on C: insering an undireced edge (denoed by InserU ), deleing an undireced edge (DeleeU), insering a direced edge (InserD), deleing a direced edge (DeleeD), making a v-srucure (MakeV) and removing a v-srucure (RemoveV). We call InserU, DeleeU, InserD, DeleeD, MakeV, and RemoveV he ypes of operaors. An operaor on a given compleed PDAG is deermined by wo pars: is ype and he modified edges. For example, he operaor InserU x y on C represens insering an undireced edge x y o C, and x y is he modified edge of he operaor. A modified graph of an operaor is he same as he iniial compleed PDAG, excep for he modified edges of he operaor. A modified graph migh (no) be a PDAG or compleed PDAG. Example 1 illusraes six operaors on a compleed PDAG C and heir corresponding modified graphs. Example 1. Figure 2 displays six operaors: InserU x z, DeleeU y z, InserD x v, DeleeD z v, MakeV z y u, and RemoveV z v u. Afer insering an undireced edge x z ino he iniial graph C, we ge a modified graph denoed as P 1 in Figure 2. By applying he oher five operaors o C in Figure 2 respecively, we can obain oher five corresponding modified graphs P 2, P 3, P 4, P 5, and P 6. Here he operaor MakeV z y u modifies z y u o z y u and he operaor Remove z v u modifies z v u o z v u. Noice ha a modified graph migh no be a PDAG hough all modified graphs in his example are PDAGs. In he above example, we see ha he modified graph of an operaor, denoed by P, migh be a PDAG, bu migh no be a compleed PDAG. For example, he modified graphs P 4, and P 6 in Figure 2 are no compleed PDAGs because he direced edge y v is no srongly proeced; and P 5 is no compleed PDAG because z y x occurs, which makes condiion (iii) in Lemma 2 no hold. A consisen exension of a PDAG P is a direced acyclic graph (DAG) on he same underlying se of edges, wih he same orienaions on he direced edges of P and he same se of v-srucures [9, 33]. According o Lemma 1, all consisen exensions of a PDAG P, if hey exis, belong o a unique Markov equivalence class. Hence if he modified graph of an operaor is a PDAG and has a consisen exension, i can resul in a compleed PDAG ha corresponds o a unique Markov equivalence class. We call i he resuling compleed PDAG of he operaor.

5 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs 5 InserU x z InserD x v MakeV z y u Iniial Compleed PDAG x z v y u (C) x z v y u (P 1 ) DeleeU y z x z v y u (P 2 ) x z v y u (P 3 ) DeleeD z v x z v y u (P 4 ) x z v y u (P 5 ) RemoveV z v u x z v y u (P 6 ) Fig 2. Examples of six operaors of PDAG C. P 1 o P 6 are he modified graphs of six operaors. To obain a valid ransiion from one compleed PDAG C o anoher, Chickering [6] inroduces he concep of validiy for an operaor on C. A valid operaor is defined as below. Definiion 3 (Valid operaor). An operaor on C is valid if (1) he modified graph of he operaor is a PDAG and has a consisen exension, and (2) all modified edges in he modified graph occur in he resuling compleed PDAG of he operaor. The firs condiion in Definiion 3 guaranees ha a valid operaor resuls in a compleed PDAG. The second condiion guaranees ha he valid operaor is effecive ; ha is, he change brough abou by he operaor occurs in he resuling compleed PDAG. The second condiion is no rivial. Consider a hree-verex compleed PDAG (x y z) ha has only one undireced edge x y; afer applying he operaor InserD y z, we ge a modified graph x y z. Clearly, x y z is a PDAG and has a consisen exension (say x y z). However, he corresponding resuling compleed PDAG is x y z and he operaor InserD y z is no effecive. Here we noice ha he second condiion is implied by he conex in Chickering [6]. Below we briefly inroduce how o obain he resuling compleed PDAG of a valid operaor from he modified graph. Verma and Pearl [33] and Meek [24] inroduce an algorihm for finding he compleed PDAG from a paern (given skeleon and v-srucures). This mehod can be used o creae he compleed PDAG from a DAG or a PDAG. They firs undirec every edge, excep for hose edges ha paricipae in a v-srucure. Then hey choose one of he undireced edges and direc i if he corresponding direced edge is srongly proeced, as shown in Figure 1 (a), (c) or (d). The algorihm erminaes when here is no undireced edge ha can be direced. Chickering [6] propose an alernaive approach o obain he compleed PDAG of a valid

6 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs 6 operaor from is modified graph. The mehod includes wo seps. The firs sep generaes a consisen exension (a DAG) of he modified graph (a PDAG) using algorihm described in Dor and Tarsi [9]. The second sep creaes a compleed PDAG corresponding o he consisen exension. We recall Dor and Tarsi a algorihm in Algorihm 3 in Appendix A. Chickering s algorihms [5] for he second sep are also summarized in Algorihm 4 in Appendix A for compleeness of he paper. The approach proposed by Chickering [5] is more complicaed bu more efficien [24] han Meek s mehod described above. Hence in his paper, we use Chickering s approach o obain he resuling compleed PDAG of a valid operaor from is modified graph. Below, we give an example o illusrae his approach. Iniial compleed PDAG x z v y u (C) (apply operaor) = modified graph x z v y u (P 6 ) consisen exension x z sep 1 = v y u (D 6 ) he resuling compleed PDAG x z sep 2 = v y u (C 1 ) Fig 3. Example for consrucing he unique resuling compleed PDAG of a valid operaor. An operaor Remove z v u in Figure 2 is applied o he iniial compleed PDAG C and finally resuls in he resuling compleed PDAG C 1. Example 2. Consider he iniial compleed PDAG C and he operaor Remove z v u in Figure 2. We illusrae in Figure 3 he seps of Chickering s approach ha generaes he resuling compleed PDAG C 1 by applying Remove z v u o C. The firs sep (sep 1) exends he modified graph (a PDAG P 6 ) o a consisen exension (D 6 ) via Algorihm 3 in Appendix A. The second sep (sep 2) consrucs he resuling compleed PDAG C 1 of he operaor Remove z v u from he DAG D 6 via Algorihm 4 in Appendix A. Wih a se of valid operaors, a Markov chain on compleed PDAGs can be consruced. Le S p be he whole se of compleed PDAGs wih p verices, S be a given subse of S p. For any compleed PDAG C S, le O C be a se of valid operaors of ineres o be defined laer on C. A se of valid operaors on S is defined as O = O C. (1.1) C S Here we noice ha each operaor in O is specific o he compleed PDAG ha he operaor applies o. A Markov chain {e } on S based on he se O can be defined as follows. Definiion 4 (A Markov Chain {e } on S). The Markov Chain {e } deermined by a se of valid operaors O is generaed as follows: sar a an arbirary compleed PDAG, denoed as e 0 = C 0 S, and repea he following seps for =0, 1,,

7 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs 7 1. A he -h sep we are a a compleed PDAG e. 2. We choose an operaor o e uniformly from O e ; if he resuling compleed PDAG C +1 of o e is in S, move o C +1 and se e +1 = C +1, oherwise we say a e and se e +1 = e. There are oher ransiions available in he lieraure. For example, he ransiions proposed by Pena [30] move from compleed PDAGs o heir modified graphs only if he modified graph happens o be is resuling compleed PDAG. Obviously, he ransiions used in Definiion 4 are superior o Pena s because hey provides more ransiion saes for any compleed PDAG. The se S is he finie sae space of chain {e }. Clearly, he sequence of compleed PDAGs {e : =0, 1, } in Definiion 4 is a discree-ime Markov chain [21, 26]. Le p CC be he one-sep ransiion probabiliy of {e } from C o C for any wo compleed PDAGs C and C in S. A Markov chain {e } is irreducible if i can reach any compleed PDAG saring a any sae in S. If {e } is irreducible, here exiss a unique saionary disribuion π =(π C, C S) saisfying balance equaions [26] π C = C S π C p C C for all C S. (1.2) An irreducible chain e is reversible if here exiss a probabiliy disribuion π such ha π C p CC = π C p C C, for all C, C S. (1.3) Sandard resuls imply ha π is he unique saionary disribuion of he discree-ime Markov chain {e } if i is finie, reversible, and irreducible. Moreover, he saionary probabiliies π C can be calculaed efficienly if he Markov chain saisfies Equaion (1.3). The properies of he Markov chain {e } given in Definiion 4 depend on he operaor se O. To implemen score-based searching in he whole se of Markov equivalence classes, Chickering [6] inroduces a se of operaors wih ypes of InserU, DeleeU, InserD, DeleeD, MakeV, or ReverseD (reversing he direcion of a direced edge), subjec o some validiy condiions. Unforunaely, he Markov chain in Definiion 4 is no reversible if he se of Chickering s operaors is used. Our goal is o design a reversible Markov chain, as i makes i easier o compue he saionary disribuion, and hereby o sudy he properies of a subse of Markov equivalence classes. In secion 2, we firs discuss he properies of an operaor se O needed o guaranee ha he Markov chain is reversible. Secion 2 also explains how o use he samples from he Markov chain o sudy properies of any given subse of Markov equivalence classes. In Secion 3 we focus on sudying ses of sparse Markov equivalence classes. Finally, in Secion 4, we repor he properies of direced edges and chain componens in sparse Markov equivalence classes wih up o one housand of verices. 2. Reversible Markov chains on Markov equivalence classes Le S be any subse of he whole se S p, and O be a se of operaors on S defined in Equaion (1.1). As in Definiion 4, we can obain a Markov chain denoed by {e }. We firs discuss four properies of O ha guaranee ha {e } is reversible and irreducible. They are

8 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs 8 validiy, disinguishabiliy, irreducibiliy and reversibiliy. We call a se of operaors perfec if i saisfies hese four properies. Then we give he saionary disribuion of {e } when O is perfec and show how o use {e } o sudy properies of S A reversible Markov chain based on a perfec se of operaors Le p CC be a one-sep ransiion probabiliy of {e } from C o C for any wo compleed PDAGs C and C in S. In order o formulae p CC clearly, we inroduce wo properies of O: Validiy and Disinguishabiliy. Definiion 5 (Validiy). Given S and any compleed PDAG C in S, a se of operaors O on S is valid if for any operaor o C (o wihou confusion below) in O C, o is valid according o Definiion 3 and he resuling compleed PDAG obained by applying o o C is also in S. According o Definiion 5, if a se of operaors O on S is valid, we can move o a new compleed PDAG in each sep of {e } and he one-sep ransiion probabiliy of any compleed PDAG o iself is zero: p CC =0, for any compleed PDAG C S. (2.1) For a se of valid operaors O and any compleed PDAG C in S, we define he resuling compleed PDAGs of he operaors in O C as he direc successors of C. For any direc successor of C, denoed by C, we obain p CC clearly as in Equaion (2.2) if O has he following propery. Definiion 6 (Disinguishabiliy). A se of valid operaors O on S is disinguishable if for any compleed PDAG C in S, differen operaors in O C will resul in differen compleed PDAGs. If O is disinguishable, for any direc successor of C, denoed by C, here is a unique operaor in O C ha can ransform C o C. Thus, he number of operaors in O C is he same as he number of direc successors of C. Sampling operaors from O C uniformly generaes a uniformly random ransiion from C o is direc successors. By denoing M(O C ) as he number of operaors in O C, we have { 1/M (OC ), C p CC = is a direc successor of C S; (2.2) 0, oherwise. We inroduce his propery because i makes compuaion of p CC efficien: if O is disinguishable, we know p CC righ away from M(O C ). In order o make sure he Markov chain {e } is irreducible and reversible, we inroduce wo more properies of O: irreducibiliy and reversibiliy. Definiion 7 (Irreducibiliy). A se of operaors O on S is irreducible if for any wo compleed PDAGs C, C S, here exiss a sequence of operaors in O such ha we can obain C from C by applying hese operaors sequenially. If O is irreducible, saring a any compleed PDAG in S, we have posiive probabiliy o reach any oher compleed PDAG via a sequence of operaors in O. Thus, he Markov chain {e } is irreducible.

9 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs 9 Definiion 8 (Reversibiliy). A se of operaors O on S is reversible if for any compleed PDAG C S and any operaor o O C wih C being he resuling compleed PDAG of o, here is an operaor o O C such ha C is he resuling compleed PDAG of o. If he se of operaors O on S is valid, disinguishable and reversible, for any pair of compleed PDAGs C, C S, C is also a direc successor of C if C is a direc successor of C. For any C S and any of is direc successors C, we have p CC =1/M (O C ) and p C C =1/M (O C ). (2.3) Le T = C S M(O C), and define a probabiliy disribuion as π C = M(O C )/T. (2.4) Clearly, Equaion (1.3) holds for π C in Equaion (2.4) if O is valid, disinguishable and reversible. π C is he unique saionary disribuion of {e } if i is also irreducible [1, 21, 26]. In he following proposiion, we summarize our resuls abou he Markov Chain {e } on S, and give is saionary disribuion. Proposiion 1 (Saionary disribuion of {e }). Le S be any given se of compleed PDAGs. The se of operaors is defined as O = C S O C where O C is a se of operaors on C for any C in S. Le M(O C ) be he number of operaors in O C. For he Markov chain {e } on S generaed according o Definiion 4, if O is perfec, ha is, he properies validiy, disinguishabiliy, reversibiliy and irreducibiliy hold for O, hen 1. he Markov chain {e } is irreducible and reversible, 2. he disribuion π C in Equaion (2.4) is he unique saionary disribuion of {e } and π C M(O C ). The challenge is o consruc a concree perfec se of operaors. In Secion 3, we carry ou such a consrucion for a se of Markov equivalence classes wih sparsiy consrains and provide algorihms o obain a reversible Markov chain. We now show ha a reversible Markov chain can be used o compue ineresing properies of a compleed PDAG se S Esimaing he properies of S by a perfec Markov chain For any C S, le f(c) be a real funcion describing any propery of ineres of C, and he random variable u be uniformly disribued on S. In order o undersand he propery of ineres, we compue he disribuion of f(u). For example, if we are ineresed in he proporion of compleed PDAGs wih a mos 10 undireced edges in S, we can define f(u) as he number of undireced edges in u and hen compue he probabiliy of {f(u) 10}. The proporion of Markov equivalence classes wih size of one (equivalenly, compleed PDAGs ha are direced) in S p is sudied in he lieraure [13, 14, 30]. For his purpose, we can define f(u) as he size of Markov equivalence classes represened by u and obain he proporion by compuing he probabiliy of {f(u) = 1}. Le A be any subse of R, he probabiliy of {f(u) A} is P (f(u) A) = {C : f(c) A, C S} S = C S I {f(c) A}, (2.5) S

10 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs 10 where S is he number of elemens in he se S and I is an indicaor funcion defined as { 1, if f(c) A; I {f(c) A} = 0, oherwise. Le {e } =1,,N be a realizaion of Markov chain {e } on S based on a perfec operaor se O according o Definiion 4 and M = M(O e ). Le π(e ) be he saionary probabiliy of Markov chain {e }. From Proposiion 1, we have π(e ) M for =1,,N. We can use {e,m } =1,,N o esimae he probabiliy of {f(u) A} by ˆP N (f(u) A) = N =1 I {f(e ) A}M 1 N =1 M 1 (2.6) From he ergodic heory of Markov chains [1, 21, 26], we can ge Proposiion 2 direcly. Proposiion 2. Le S be a given se of compleed PDAGs, and assume he se of operaors O on S is perfec. The Markov chain {e } =1,,N is obained according o Definiion 4. Then, he esimaor ˆP N ({f(u) A}) in Equaion (2.6) converges o P ({f(u) A}) in Equaion (2.5) wih probabiliy one, ha is, ) P (ˆPN (f(u) A) P (f(u) A) as N =1. (2.7) Proposiion 2 shows ha he esimaor defined in Equaion (2.6) is a consisen esimaor of P (f(u) A). We can sudy any given subse of Markov equivalence classes via Equaion (2.6) if we can obain {e } =1,,N and {M } =1,,N. We now urn o consruc a concree perfec se of operaors for a se of compleed PDAGs wih sparsiy consrains and hen inroduce algorihms o run a reversible Markov chain. 3. A Reversible Markov chain on compleed PDAGs wih sparsiy consrains We define a se of Markov equivalence classes Sp n follows: wih p verices and a mos n edges as S n p = {C : C is a compleed PDAG wih p verices and n C n}, (3.1) where n C is he number of edges in C. Recall ha S p denoes he whole se of compleed PDAGs wih p verices. Clearly, Sp n = S p when n p(p 1)/2. We now consruc a perfec se of operaors on Sp n. Noice ha our consrucions can be exended o adap o some oher ses of compleed PDAGs, say, a se of compleed PDAGS wih a given maximum degree. In Secion 3.1, we consruc he perfec se of operaors for any compleed PDAG in Sp n. In Secion 3.2, we propose algorihms and heir acceleraed version for efficienly obaining a Markov chain based on he perfec se of operaors. We finally provide he proofs of all heorems in his secion in Appendix B.

11 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs Consrucion of a perfec se of operaors on S n p In order o consruc a perfec se of operaors, we need o define he se of operaors on each compleed PDAG in S n p. Le C be a compleed PDAG in S n p. We consider six ypes of operaors on C ha were inroduced in Secion 1.2: InserU, DeleeU, InserD, DeleeD, MakeV, and RemoveV. The operaors on C wih he same ype bu differen modified edges consiue a se of operaors. We inroduce six ses of operaors on C denoed by InserU C, DeleeU C, InserD C, DeleeD C, MakeV C and RemoveV C in Definiion 9. Firs we explain some noaion used in Definiion 9. Le x and y be any wo disinc verices in C. The neighbor se of x denoed by N x consiss of every verex y wih x y in C. The common neighbor se of x and y is defined as N xy = N x N y. x is a paren of y and y is a child of x if x y occurs in C. A verex u is a common child of x and y if u is a child of boh x and y. Π x represens he se of all parens of x. Definiion 9 (Six ses of operaors on C). Le C be a compleed PDAG in Sp n and n C be he number of edges in C. We inroduce six ses of operaors on C: InserU C DeleeU C, InserD C, DeleeD C, MakeV C and RemoveV C as follows. (a) For any wo verices x, y ha are no adjacen in C, he operaor InserU x y on C is in InserU C if and only if (iu 1 ) n C <n; (iu 2 ) InserU x y is valid; (iu 3 ) for any u ha is a common child of x, y in C, boh x u and y u occur in he resuling compleed PDAG of InserU x y. (b) For any undireced edge x y in C, he operaor DeleeU x y on C is in DeleeU C if and only if (du 1 ) DeleeU x y is valid. (c) For any wo verices x, y ha are no adjacen in C, he operaor InserD x y on C is in InserD C if and only if (id 1 ) n C <n; (id 2 ) InserD x y is valid; (id 3 ) for any u ha is a common child of x, y in C, y u occurs in he resuling compleed PDAG of InserD x y. (d) For any direced edge x y in C, operaor DeleeD x y on C is in DeleeD C if and only if (dd 1 ) DeleeD x y is valid; (dd 2 ) for any v ha is a paren of y bu no a paren of x, direced edge v y in C occurs in he resuling compleed PDAG of DeleeD x y. (e) For any subgraph x z y in C, he operaor MakeV x z y on C is in MakeV C if and only if (mv 1 ) MakeV x z y is valid. (f) For any v-srucure x z y of C, he operaor RemoveV x z y on C is in RemoveV C if and only if (rv 1 )Π x =Π y ; (rv 2 )Π x N xy =Π z \{x, y}; (rv 3 ) every undireced pah beween x and y conains a verex in N xy. Muneanu and Bendou [25] discuss he consrains for he firs five ypes of operaors such ha each one can ransform one compleed PDAG o anoher. Chickering [6] inroduces he necessary and sufficien condiions such ha hese five ypes of operaors are valid. The condiions proposed by Muneanu and Bendou [25] and Chickering [6] guaranee ha he condiions iu 2, du 1, id 2, dd 1 and mv 1 in Definiion 9 hold. We employ he condiions inroduced by Chickering [6] and lis hem in Lemma 3 in Appendix A. However, given only Chickering s condiions, we do no necessarily have a perfec se of operaors. We inroduce several addiional condiions in Definiion 9, including iu 1, iu 3, id 1, id 3 and dd 2 for operaors in InserU C, InserD C and DeleeD C, and rv 1,rv 2 and rv 3 for operaors in RemoveV C.

12 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs 12 The se of operaors on C denoed by O C is defined as follows. O C = InserU C DeleeU C InserD C DeleeD C MakeV C RemoveV C. (3.2) Taking he union over all compleed PDAGs in Sp n, we define he se of operaors on Sp n as O = O C, (3.3) C S n p where O C is he se of operaors in Equaion (3.2). In main resul of his paper, we show ha O in Equaion (3.3) is a perfec se of operaors on S n p. Theorem 1 (A perfec se of operaors on S n p ). O defined in Equaion (3.3) is a perfec se of operaors on S n p. The proof of Theorem 1 shows ha iu 3, id 3 and dd 2 are key condiions in Definiion 9 o guaranee ha O is reversible. We also use Example 3 o show heurisically ha condiions iu 3 and dd 2 are necessary for O o be a perfec se of operaors. Example 3. This example illusraes ha O in Equaion (3.3) will no be reversible if condiion iu 3 or dd 2 is no conained in Definiion 9. Consider operaor se O defined in Equaion (3.3) for S 5 and he compleed PDAG C S 5 in Figure 4. We have ha operaor InserU z u and DeleeD z v are valid according o Definiion 5. As shown in Figure 4, InserU z u ransfers C o he compleed PDAG C 1 and DeleeD z v ransfers C o he compleed PDAG C 2. However, deleing z u from C 1 will resul in an undireced PDAG disinc from C and InserD z v is no valid for C 2 according o Lemma 3 in Appendix A. As a consequence, if O conains InserU z u and DeleeD z v, i will be no reversible according o Definiion 8. According o Definiion 9, hese wo operaors do no appear in O C because hey do no saisfy he condiions iu 3 and dd 2 respecively. x z v y u (C 1 ) InserU z u === x z v y u (C) DeleeD z v === x z v y u (C 2 ) Fig 4. Example: Two valid operaors bring abou irreversibiliy. I shows valid condiions are no sufficien for perfec operaor se. Now we give a oy example o show how o consruc a concree perfec se of operaors following Definiion 9. Example 4. Consider he compleed PDAG C in Example 3. Here we inroduce he procedure o deermine InserU C. All possible operaors of insering an undireced edge o C include: InserU x z, InserU x u, InserU x v and InserU z u. The operaor InserU x v is no valid according o Lemma 3 in Appendix A since Π(x) Π(v). The operaor InserU z u is valid; however, condiion iu 3 does no hold according o Example

13 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs According o Definiion 9, we have ha only InserU x z and InserU x u are in InserU C. Thus InserU C = {x z, x u}, where x z denoes InserU x z in he se. Table 1 liss he six ses of operaors on C. x z v y u (C) Table 1 The six ses of operaors of C. These operaors are perfec. InserU C = {x z, x u} DeleeU C = {x y, y z, y u} InserD C = {x u} DeleeD C = {y v} MakeV C = {x y z, x y u, z y u} RemoveV C = {u v z} The preceding secion showed how o consruc a perfec se of operaors. Based on he perfec se of operaors we can obain a finie irreducible reversible discree-ime chain. In he nex subsecion, we provide deailed algorihms for obaining a Markov chain on S n p and heir acceleraed version Algorihms In his subsecion, we provide he algorihms in deail o generae a Markov chain on S n p, defined in Definiion 4 based on he perfec se of operaors defined in Equaion (3.3). A skech of Algorihm 1 is shown below; some seps of his algorihm are furher explained in he subsequen algorihms. Algorihm 1: Road map o consruc a Markov chain on S n p Inpu: p, he number of verices; n, he maximum number of edges; N, he lengh of Markov chain. Oupu: {e,m } =1,,N, where {e } is Markov chain and M is he number of operaors in O e. 1 Iniialize e 0 as any compleed PDAG in S n p ; 2 for 0 o N do Sep A Consruc he se of operaors O e in Equaion (3.2) via Algorihm 1.1; Sep B Le M be he number of operaors in O e ; Sep C Randomly choose an operaor o uniformly from O e ; Sep D Apply operaor o o e. Se e +1 as he resuling compleed PDAG of o. 3 reurn {e,m } =1,,N. Sep A of Algorihm 1 consrucs he ses of operaors on compleed PDAGs in he chain {e }. I is he mos difficul sep and dominaes he ime complexiy of Algorihm 1. Sep B and Sep C can be implemened easily afer O e is obained. Sep D can be implemened via Chickering s mehod [6] ha was menioned in Secion 1.2. We will show ha he ime complexiy of obaining a Markov chain on Sp n wih lengh N ({e } =1,,N ) is approximae O(Np 3 ) if n is he same order of p. For large p, we also provide an acceleraed version ha, in some cases, can run hundreds of imes faser.

14 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs 14 The res of his secion is arranged as follows. In Secion 3.2.1, we firs inroduce he algorihms o implemen Sep A. In Secion we discuss he ime complexiy of our algorihm, and provide an acceleraion mehod o speed up Algorihm Implemenaion of Sep A in Algorihm 1 A deailed implemenaion of Sep A (o consruc O e ) is described in Algorihm 1.1. To consruc O e in Algorihm 1.1, we go hrough all possible operaors on e and choose hose saisfying he corresponding condiions in Definiion 9. The condiions in Algorihm 1.1 include: iu 1, iu 3, id 3, dd 2, rm 1, rv 1, rv 2 iu 2.1, iu 2.2, du 1.1, id 1, id 2.1, id 2.2, id 2.3, dd 1.1 and mv 1.1. The condiions iu 1, iu 3, id 1, id 3, dd 2, rm 1, rv 1, rv 2 can be found in Definiion 9. The oher condiions (iu 2.1, iu 2.2 ), du 1.1,(id 2.1, id 2.2, id 2.3 ), dd 1.1 and mv 1.1 are equivalen, respecively, o he validiy condiions iu 2, du 1, id 2, dd 1 and mv 1 in Definiion 9 and can be found in Lemma 3 in Appendix A. For each possible operaor, we check he corresponding condiions shown in Algorihm 1.1 one-by-one unil one of hem fails. Below, we inroduce how o check hese condiions. Algorihm 1.1: Consruc he operaor se O e for a compleed PDAG e. Inpu: A compleed PDAG e wih p verices. Oupu: operaor se O e. // Direced-edges e, Undireced-edges e, Pairs-nonadj e, V-srucures e, Undireced-v-srucures e used below are ses of possible modified edges of e. Undireced-v-srucures e is he se of all subgraphs like x y z wih x and z no adjacen in e ; Pairs-nonadj e is he se of all pairs of verices ha are nonadjacen in e. 1 Se O e as empy se 2 for each undireced edge x y in Undireced-edges e do 3 consider operaor DeleeU x x, add i o O e if du 1.1 holds, 4 for each direced edge x y in Direced-edges e do 5 consider operaor DeleeD x x, add i o O e if boh dd 1.1 and dd 2 hold (dd 2 is checked in Algorihm 1.1.3) 6 for each v-srucure x z y in V-srucures e do 7 consider operaor RemoveV x k x i x l, add i o O e if rv 1, rv 2 and rv 3 hold, 8 for each undireced v-srucure x z y in Undireced-v-srucures e do 9 consider operaor MakeV x k x i x l, add i o O e if mv 1.1 holds, 10 if n e <n(i.e., iu 1 or id 1 holds) hen 11 for each pair (x, y) in Pairs-nonadj e do 12 consider operaor InserU x y, add i o O e if iu 1, iu 2.1, iu 2.2 and iu 3 hold (iu 3 is checked in Algorihm 1.1.1); 13 consider InserD x y, add i o O e, if id 1, id 2.1, id 2.2, id 2.3, and id 3 hold (id 3 is checked in Algorihm 1.1.2); 14 consider InserD x y, add i o O e if id 1, id 2.1, id 2.2, id 2.3, and id 3 hold. 15 reurn O e All condiions in Algorihm 1.1 can be classified ino four groups: (1) wheher wo verex ses are equal or no (iu 2.1, id 2.1, rv 1, and rv 2 ), (2) wheher a subgraph is a clique or no(du 1.1, id 2.2, and dd 1.1 ), (3) wheher all parially direced pahs or all undireced pahs

15 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs 15 beween wo verices conain a leas one verex in a given se (iu 2.2, id 2.3, mv 1.1 and rv 3 ), and (4) ohers (iu 3, id 3, and dd 2 ). The condiions in he firs hree groups can be esed via classical graph algorihms; we briefly review hese below. Checking he condiions in he firs wo groups is rivial and very efficien, because he ses involved are small for mos compleed PDAGs in Sp n when n is of he same order of p. To check he condiions in he hird group, we jus need o check wheher here is a parially direced pah or undireced pah beween wo given verices no hrough any verices in he given se. We check his using a deph-firs search from he source verex. When looking for an undireced pah, we can search wihin he corresponding chain componen ha includes boh he source and he desinaion verex. The condiions iu 3, id 3 and dd 2 in he fourh group depend on boh e and he resuling compleed PDAGs of he operaors. Inuiively, checking hese hree condiions requires ha we obain he corresponding resuling compleed PDAGs. We know ha he ime complexiy of geing a resuling compleed PDAG of e is O(pn e )[6, 9], where n e is he number of edges in e. To avoid generaing resuling compleed PDAG, we provide hree algorihms o check iu 3, id 3 and dd 2 respecively. In hese hree algorihms, we use he concep of srongly proeced edges, defined in Definiion 2. Le v conain all verices adjacen o v. To check wheher a direced edge v u is srongly proeced or no in a graph G, from Definiion 2, we need o check wheher one of he four configuraions in Figure 1 occurs in G. This can be implemened by local search in v and u. We know ha when a PDAG is sparse, in general, hese ses are small, so i is very efficien o check wheher an edge is srongly proeced. We are now ready o provide Algorihm 1.1.1, Algorihm 1.1.2, and Algorihm o check iu 3, id 3 and dd 2 only based on e, respecively. In hese hree algorihms, we jus need o check wheher a few direced edges are srongly proeced or no in P +1, which has only one or a few edges differen from e. We prove in Theorem 2 ha hese hree algorihms are equivalen o checking condiions iu 3, id 3 and dd 2, respecively. Algorihm 1.1.1: Check he condiion iu 3 in Definiion 9 Inpu: a compleed PDAG e and a valid operaor on i: InserU x y. Oupu: True or False 1 Inser x y o e, ge he modified PDAG denoed as P +1, 2 for each common child u of x and y in P +1 do 3 if eiher x u or y u is no srongly proeced in P +1 hen 4 reurn False 5 reurn True (iu 3 holds for InserU x y) Theorem 2 (Correcness of Algorihms 1.1.1, and 1.1.3). Le e be a compleed PDAG. We have he following resuls. (i) Le InserU x y be any valid operaor of e, hen condiion iu 3 holds for he operaor InserU x y if and only if he oupu of Algorihm is True. (ii) Le InserD x y be any valid operaor of e, hen condiion id 3 holds for he operaor InserD x y if and only if he oupu of Algorihm is True.

16 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs 16 Algorihm 1.1.2: Check he condiion id 3 in Definiion 9 Inpu: a compleed PDAG e and a valid operaor: InserD x y. Oupu: True or False 1 Inser x y o e, ge a PDAG, denoed as P o, 2 for each undireced edge u y in P o, where u is no adjacen o x do 3 updae P o by oriening u y o y u, 4 for each edge v y in P o do 5 if v y is no srongly proeced in P o hen 6 updae P o by changing v y o v y, 7 Se P +1 = P o 8 for each common child u of x and y in P +1 do 9 if y u is no srongly proeced in P +1 hen 10 reurn False 11 reurn True (id 3 holds for InserD x y) Algorihm 1.1.3: Check he condiion dd 2 in Definiion 9 Inpu: a compleed PDAG e and a valid operaor DeleeD x y Oupu: True or False 1 Delee x y from e, ge a PDAG, denoed as P +1 ; 2 for each paren v of y in P +1 do 3 if v y is no srongly proeced in P +1 hen 4 reurn False 5 reurn True (dd 2 holds for DeleeD x y) (iii) Le DeleeD x y be any valid operaor of e, hen condiion dd 2 holds for he operaor DeleeD x y if and only if he oupu of Algorihm is True. In Theorem 2, we show ha an algorihm (Algorihm 1.1.1, Algorihm 1.1.2, or Algorihm 1.1.3) reurns True for an operaor if and only if he corresponding condiion (iu 3, id 3 or dd 2 ) holds for he operaor. Theorem 2 says ha we do no have o examine he resuling compleed PDAG o check condiions iu 3, id 3 and dd 2, which saves much compuaion ime Time complexiy of Algorihm 1 and an acceleraed version We now discuss he ime complexiy of Algorihm 1. For e Sp n, le p and n be he number of verices and edges in e respcively, k be he number of v-srucures in e, and k be he number of undireced v-srucures (subgraphs x y z wih x and z nonadjacen) in e. To consruc O e, in Sep A of Algorihm 1 (equivalen, Algorihm 1.1), all possible operaors we need o go hrough include: n deleing operaors (DeleeU and DeleeD), 3(p(p 1)/2 n ) insering operaors (InserU and InserD) when he number of edges in e is less han n, k RemoveV operaors and k MakeV operaors. There are a mos Q =1.5p(p 1) 2n +k +k possible operaors for e. Among all condiions in Algorihm 1.1, he mos ime-consuming one, which akes ime O(p + n )[6], is o look for a pah via he deph-firs search for an operaor wih ype of InserD. We have ha he ime complexiy of consrucing O e in

17 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs 17 Algorihm 1.1 is O(Q (p + n )) in he wors case and he ime complexiy of Algorihm 1 is O( N =1 Q (p + n )) in he wors case, where N is he lengh of Markov chain in Algorihm 1. We know ha k and k reach he maxima (p 2)/2 floor(p/2) ceil(p/2) when e is a evenly divided complee biparie graphs [14]. Consequenly, he ime complexiy of Algorihm 1 are O(Np 4 ) in he wors case. Forunaely, when n is a few imes of p, say n =2p, all compleed PDAGs in Sp n are sparse and our experimens show k and k are much less han O(p 2 ) for mos compleed PDAGs in Markov chain {e } =1,,N. Hence he ime complexiy of Algorihm 1 is approximae O(Np 3 ) on average when n is a few imes of p. We can implemen Algorihm 1 efficienly when p is no large (less or around 100 in our experimens). However, when p is larger, we need large N o guaranee he esimaes reach convergence. Experimens in Secion 4 show N = 10 6 is suiable. In his case, cubic complexiy (O(Np 3 )) of Algorihm 1 is unaccepable. We need o speed up he algorihms for a very large p. Noice ha in Algorihm 1, we obain an irreducible and reversible Markov chain {e } and a sequence of numbers {M } by checking all possible operaors on each e. The sequence {M } are used o compue he saionary probabiliies of {e } according o Proposiion 1. We now inroduce an acceleraed version of Algorihm 1 o generae irreducible and reversible Markov chains on Sp n. The basic idea is ha we do no check all possible operaors bu check some random samples. These random samples are hen used o esimae {M }. We firs explain some noaion used in he acceleraed version. For each compleed PDAG e, if n e <n, O e (all) is he se of all possible operaors on e wih ypes of InserU, DeleeU, InserD, DeleeD, MakeV, and RemoveV. If n e = n, he number of edges in e reaches he upper bound n, no more edges can be insered ino e. Le O e ( inser) be he se of operaors obained by removing operaors wih ypes of InserU and InserD from O e (all). O e ( inser) is he se of all possible operaors on e when n e = n. We can obain O e (all) and O e ( inser) easily via all possible modified edges inroduced in Algorihm 1.1. The acceleraed version of Algorihm 1 is shown in Algorihm 2. In Algorihm 2, O e (eiher O e (all) or O e ( inser) ) is he se of all possible operaors on e, α (0, 1] is an acceleraion parameer ha deermine how many operaors in O e are checked, O e (check) is a se of checked operaors ha are randomly sampled wihou replacemen from O e, and Õe is he se of all perfec operaors in O e (check). When α = 1, Õe = O e and algorihm 2 becomes back o Algorihm 1. In Algorihm 2, because he operaors in Õe are i.i.d. sampled from O e in Sep A and operaor o is chosen uniformly from Õe in Sep C, clearly, o is also chosen uniformly from O e. We have ha he following Corollary 1 holds according o Proposiion 1. Corollary 1 (Saionary disribuion of {e } on Sp n ). Le Sp n, defined in Equaion (3.1), be he se of compleed PDAGs wih p verices and maximum n of edges, O e, defined in Equaion (3.2), be he se of operaors on e, and M be he number of operaors in O e. For he Markov chain {e } on Sp n obained via Algorihm 1 or Algorihm 2, hen 1. he Markov chain {e } is irreducible and reversible; 2. he Markov chain {e } has a unique saionary disribuion π and π(e ) M. In Algorihm 2, we provide an esimae of M insead of calculaing i exacly in Algorihm

18 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs 18 Algorihm 2: An acceleraed version of Algorihm 1. Inpu: α (0, 1]:an acceleraion parameer; p, n and N, he same as inpu in Algorihm 1 Oupu: {e, ˆM } =1,,N, where ˆM is an esimaion of M = O e 1 Iniialize e 0 as any compleed PDAG in S n p ; 2 for 0 o N do 3 Sep A : 4 if n e <nhen 5 Se O e = O (all) e 6 else 7 Se O e = O ( inser) e 8 Se m = O e 9 Randomly sample [αm ] operaors wihou replacemen from O e o generae a se O e (check), where [αm ] is he ineger closes o αm. 10 Check all operaors in O e (check) and choose hose saisfying he corresponding condiions in Definiion 9 o consruc a se of operaors Õe. /* The deailed o check condiions can be found in Secion */ 11 Se m (Õ) 12 Sep B : 13 Le ˆM m (Õ) = m [αm ], 14 Sep C : = Õe. If m(õ) = 0, goo Line Randomly choose an operaor o uniformly from Õe. 16 Sep D: 17 Apply operaor o o e. Se e +1 as he resuling compleed PDAG of o. 18 reurn {e, ˆM } =1,,N. 1. Le O e = m, O (check) e =[αm ], and Õe = m (Õ). Clearly, he raio m (Õ) /[αm ] is an unbiased esimaor of he populaion proporion M /m via sampling wihou replacemen. We can esimae M = O e in Sep B as m ˆM (Õ) = m [αm ]. (3.4) We have ha when [αm ] is large, he esimaor ˆM has an approximae normal disribuion wih mean equal o M = O e. Le he random variable u be uniformly disribued on Sp n, f(u) be a real funcion describing a propery of ineres of u, and A be a subse of R. By replacing M wih ˆM in Equaion (2.6), we esimae P N ({f(u) A}) via {e, ˆM } =1,,N as follows. ˆP N (f(u) A) = where P N (f(u) A) is defined in Equaion (2.5). N =1 I {f(e ) A} N =1 1 ˆM ˆM 1, (3.5)

19 Y.B. He e al./reversible MCMC on markov equivalence classes of sparse DAGs 19 In he acceleraed version, only 100α% of all possible operaors on e are checked. In Secion 4, our experimens on S show ha he acceleraed version can speed up he approach nearly 1 α imes, and ha Equaion (3.5) provide almos he same resuls as Equaion (2.6) in which {e,m } =1,,N from Algorihm 1 are used. Roughly speaking, if we se α =1/p, he ime complexiy of our acceleraed version can reduce o O(Np 2 ). 4. Experimens In his secion, we conduc experimens o illusrae he reversible Markov chains proposed in his paper and heir applicaions for sudying Markov equivalence classes. The main poins obained from hese experimens are as follows. 1. For S p wih small p, our proposed approach is housands of imes faser han available mehods for sudying ses of Markov equivalence classes in he lieraure, such as hose proposed by Gillespie and Perlman [14], or Pena [30]. For Sp n wih large p (up o 1000), he acceleraed version of our proposed approach is also very efficien and he esimaions in Equaions (2.6) and (3.5) converge quickly as he lengh of Markov chain increases. 2. For compleed PDAGs in Sp n wih sparsiy consrain (n is a small muliple of p), we see ha (i) mos edges are direced, (ii) he sizes of maximum chain componens (measured by he number of verices) are very small (around en) even for large p (around 1000), and (iii) he number of chain componens grows approximaely linearly wih p. These resuls imply ha, under he assumpion ha he underlying compleed PDAG is sparse, and ha here are no laen or selecion variables presen, causal inference based on observaional daa is sufficien o recover mos causal relaionships. Moreover, under hese assumpions, mos chain componens are small, so in general few inervenions are needed o infer he direcions of he undireced edges. In Secion 4.1, we evaluae our mehods by comparing he size disribuions of Markov equivalence classes in S p wih small p o rue disribuions (p =3, 4) or Gillispie s resuls (p = 6) [14]. In Secion 4.2, we repor he proporion of direced edges and he properies of chain componens of Markov equivalence classes under sparsiy consrains. In Secion 4.3, we show experimenally ha Algorihm 2 is much faser han Algorihm 1, and ha he difference in he esimaes obained is small. Finally, we sudy he asympoic properies of our proposed esimaors in Secion Size disribuions of Markov Equivalence classes in S p for small p We consider size disribuions of compleed PDAGs in S p for p =3, 4, and 6 respecively. There are 11 Markov equivalence classes in S 3, and 185 Markov equivalence classes in S 4. Here we can ge he rue size disribuions for S 3 and S 4 by lising all he Markov equivalence classes and calculaing he size of each explicily. Gillespie and Perlman calculae he rue size probabiliies for S 6 by lising all classes; hese are denoed as GP-values. We esimae he size probabiliies via Equaion (2.6) wih he Markov chains from Algorihm 1. We ran en independen Markov Chains using Algorihm 1 o calculae he mean and sandard deviaion of each esimae. The resuls are shown in Table 2, where N is he sample size (lengh of