REVERSIBLE MCMC ON MARKOV EQUIVALENCE CLASSES OF SPARSE DIRECTED ACYCLIC GRAPHS 1

Transcription

1 The Annals of Saisics 2013, Vol. 41, No. 4, DOI: /13-AOS1125 Insiue of Mahemaical Saisics, 2013 REVERSIBLE MCMC ON MARKOV EQUIVALENCE CLASSES OF SPARSE DIRECTED ACYCLIC GRAPHS 1 BY YANGBO HE, JINZHU JIA AND BIN YU Peking Universiy, Peking Universiy and Universiy of California, Berkeley Graphical models are popular saisical ools which are used o represen dependen or causal complex sysems. Saisically equivalen causal or direced graphical models are said o belong o a Markov equivalen class. I is of grea ineres o describe and undersand he space of such classes. However, wih currenly known algorihms, sampling over such classes is only feasible for graphs wih fewer han approximaely 20 verices. In his paper, we design reversible irreducible Markov chains on he space of Markov equivalen classes by proposing a perfec se of operaors ha deermine he ransiions of he Markov chain. The saionary disribuion of a proposed Markov chain has a closed form and can be compued easily. Specifically, we consruc a concree perfec se of operaors on sparse Markov equivalence classes by inroducing appropriae condiions on each possible operaor. Algorihms and heir acceleraed versions are provided o efficienly generae Markov chains and o explore properies of Markov equivalence classes of sparse direced acyclic graphs (DAGs) wih housands of verices. We find experimenally ha in mos Markov equivalence classes of sparse DAGs, (1) mos edges are direced, (2) mos undireced subgraphs are small and (3) he number of hese undireced subgraphs grows approximaely linearly wih he number of verices. 1. Inroducion. Graphical models based on direced acyclic graphs (DAGs, denoed as D) are widely used o represen causal or dependen relaionships in various scienific invesigaions, such as bioinformaics, epidemiology, sociology and business [12, 13, 19, 20, 24, 32, 35]. A DAG encodes he independence and condiional independence resricions of variables. However, because differen DAGs can encode he same se of independencies or condiional independencies, mos of he ime we canno disinguish DAGs via observaional daa [31]. Received Sepember 2012; revised April Suppored in par by NSFC ( , , ), 973 Program-2007CB814905, DPHEC , US NSF Grans DMS , DMS , DMS , DMS , SES (CDI), US ARO gran W911NF and he Cener for Science of Informaion (CSoI), a US NSF Science and Technology Cener, under Gran agreemen CCF This research was also suppored by School of Mahemaical Science, he Cener of Saisical Sciences, he Key Lab of Mahemaical Economics and Quaniaive Finance (Minisry of Educaion), he Key lab of Mahemaics and Applied Mahemaics (Minisry od Educaion), and he Microsof Join Lab on Saisics and informaion echnology a Peking Universiy. MSC2010 subjec classificaions. 62H05, 60J10, 05C81. Key words and phrases. Sparse graphical model, reversible Markov chain, Markov equivalence class, Causal inference. 1742

2 REVERSIBLE MCMC ON MECS OF SPARSE DAGS 1743 A Markov equivalence class is used o represen all DAGs ha encode he same dependencies and independencies [2, 6, 33]. A Markov equivalence class can be visualized (or modeled) and uniquely represened by a compleed parial direced acyclic graph (compleed PDAG for shor) [6] which possibly conains boh direced edges and undireced edges [22]. There exiss a one-o-one correspondence beween compleed PDAGs and Markov equivalence classes [2]. The compleed PDAGs are also called essenial graphs by Andersson e al. [2] and maximally oriened graphs by Meek [26]. A se of compleed PDAGs can be used as a model space. The modeling ask is o discover a proper Markov equivalence class in he model space [3, 4, 8, 9, 18, 25]. Undersanding he se of Markov equivalence classes is imporan and useful for saisical causal modeling [14, 15, 21]. For example, if he number of DAGs is large for Markov equivalence classes in he model space, searching based on unique compleed PDAGs could be subsanially more efficien han searching based on DAGs [6, 25, 27]. Moreover, if mos compleed PDAGs in he model space have many undireced edges (wih nonidenifiable direcions), many inervenions migh be needed o idenify he causal direcions [11, 17]. Because he number of Markov equivalence classes increases superexponenially wih he number of verices (e.g., more han classes wih 10 verices) [15], i is hard o sudy ses of Markov equivalence classes. To our knowledge, only compleed PDAGs wih a small given number of verices ( 10) have been sudied horoughly in he lieraure [14, 15, 29]. Moreover, hese sudies focus on he size of Markov equivalence classes, which is defined as he number of DAGs in a Markov equivalence class. Gillispie and Perlman [15] obain he rue size disribuion of all Markov equivalence classes wih a given number (10 or fewer) of verices by lising all classes. Peña [29] designs a Markov chain o esimae he proporion of he equivalence classes conaining only one DAG for graphs wih 20 or fewer verices. In recen years, sparse graphical models have become popular ools for fiing high-dimensional mulivariae daa. The sparsiy assumpion inroduces resricions on he model space; a sandard resricion is ha he number of edges in he graph be less han a small muliple of he number of verices. I is hus boh ineresing and imporan o be able o explore he properies of subses of graphical models, especially wih sparsiy consrains on he edges. In his paper, we propose a reversible irreducible Markov chain on Markov equivalence classes. We firs inroduce a perfec se of operaors ha deermine he ransiions of he chain. Then we obain he saionary disribuion of he chain by couning (or esimaing) all possible ransiions for each sae of he chain. Finally, based on he saionary disribuion of he chain (or esimaed saionary disribuion), we re-weigh he samples from he chain. Hence hese reweighed samples can be seen as uniformly (or approximaely uniformly) generaed from he Markov equivalence classes of ineres. Our proposal allows he sudy of properies of he ses ha conain sparse Markov equivalence classes in a compuaionally efficien manner for sparse graphs wih housands of verices.

3 1744 Y. HE, J. JIA AND B. YU 1.1. A Markov equivalence class and is represenaion. In his secion, we give a shor overview for he represenaions of a Markov equivalence class. AgraphG is defined as a pair (V, E), wherev ={x 1,...,x p } denoes he verex se wih p variables, and E denoes he edge se. Le n G = E be he number of edges in G. A direced (undireced) edge is denoed as or ( ). A graph is direced (undireced) if all of is edges are direced (undireced). A sequence (x 1,x 2,...,x k ) of disinc verices is called a pah from x 1 o x k if eiher x i x i+1 or x i x i+1 is in E for all i = 1,...,k 1. A pah is parially direced if a leas one edge in i is direced. A pah is direced (undireced) if all edges are direced (undireced). A cycle is a pah from a verex o iself. A direced acyclic graph (DAG), denoed by D, is a direced graph which does no conain any direced cycle. Le τ be a subse of V.Thesubgraph D τ = (τ, E τ ) induced by he subse τ has verex se τ and edge se E τ, he subse of E which conains he edges wih boh verices in τ. A subgraph x z y is called a v- srucure if here is no edge beween x and y. A parially direced acyclic graph (PDAG), denoed by P, is a graph wih no direced cycle. A graphical model consiss of a DAG and a join probabiliy disribuion. Wih he graphical model, in general, he condiional independencies implied by he join probabiliy disribuion can be read from he DAG. A Markov equivalence class (MEC) is a se of DAGs ha encode he same se of independencies or condiional independencies. Le he skeleon of an arbirary graph G be he undireced graph wih he same verices and edges as G, regardless of heir direcions. Verma and Pearl [36] proved he following characerizaion of Markov equivalence classes: LEMMA 1 (Verma and Pearl [36]). Two DAGs are Markov equivalen if and only if hey have he same skeleon and he same v-srucures. This lemma implies ha, among DAGs in an equivalence class, some edge orienaions may vary, while ohers will be preserved (e.g., hose involved in a v- srucure). Consequenly, a Markov equivalence class can be represened uniquely by a compleed PDAG, defined as follows: DEFINITION 1 (Compleed PDAG [6]). The compleed PDAG of a DAG D, denoed as C, is a PDAG ha has he same skeleon as D, and an edge is direced in C if and only if i has he same orienaion in every equivalen DAG of D. According o Definiion 1 and Lemma 1, a compleed PDAG of a DAG D has he same skeleon as D, and i keeps a leas he direced edges ha occur in he v-srucures of D. Anoher popular name of a compleed PDAG is essenial graph inroduced by Andersson e al. [2], who inroduce four necessary and sufficien condiions for a graph o be an essenial graph; see hem in Lemma 2, Appendix A.1. One of he condiions shows ha all direced edges in a compleed PDAG mus be srongly proeced, defined as follows:

4 REVERSIBLE MCMC ON MECS OF SPARSE DAGS 1745 FIG. 1. Four configuraions where v u is srongly proeced in G. DEFINITION 2. Le G = (V, E) be a graph. A direced edge v u E is srongly proeced in G if v u E occurs in a leas one of he four induced subgraphs of G in Figure 1. If we delee all direced edges from a compleed PDAG, we are lef wih several isolaed undireced subgraphs. Each isolaed undireced subgraph is a chain componen of he compleed PDAG. Observaional daa is no sufficien o learn he direcions of undireced edges of a compleed PDAG; one mus perform addiional inervenion experimens. In general, he size of a chain componen is a measure of complexiy of causal learning; he larger he chain componens are, he more inervenions will be necessary o learn he underlying causal graph [17]. In learning graphical models [6] or sudying Markov equivalence classes [29], Markov chains on compleed PDAGs play an imporan role. We briefly inroduce he exising mehods o consruc Markov chains on compleed PDAGs in he nex subsecion Markov chains on compleed PDAGs. To consruc a Markov chain on compleed PDAGs, we need o generae he ransiions among hem. In general, an operaor ha can modify he iniial compleed PDAG locally can be used o carry ou a ransiion [6, 27, 29, 34]. Le C be a compleed PDAG. We consider six ypes of operaors on C: insering an undireced edge (denoed by InserU ), deleing an undireced edge (DeleeU), insering a direced edge (InserD), deleing a direced edge (DeleeD), making a v-srucure (MakeV) and removing a v-srucure (RemoveV). We call InserU, DeleeU, InserD, DeleeD, MakeV and RemoveV he ypes of operaors. An operaor on a given compleed PDAG is deermined by wo pars: is ype and he modified edges. For example, he operaor InserU x y onc represens insering an undireced edge x y o C, andx y is he modified edge of he operaor. A modified graph of an operaor is he same as he iniial compleed PDAG, excep for he modified edges of he operaor. A modified graph migh (no) be a compleed PDAG; see Example 1 in Secion 2.1, of he Supplemenary Maerial [16]. Madigan e al. [25], Perlman [34] and Peña [29] inroduce several Markov chains based on he modified graphs of operaors. A each sae of hese Markov chains, say C, hey move o he modified graph of an operaor on C only when he modified graph happens o be a compleed PDAG, oherwise, say a C. Inorder o move o new compleed PDAGs, Madigan e al. [25] search he operaors

5 1746 Y. HE, J. JIA AND B. YU whose modified graphs are compleed PDAG by checking Andersson s condiions [2] one by one. Perlman [34] inroduces an alernaive search approach ha is more efficien by exploiing furher Andersson s condiions. When he modified graph of an operaor on C is no a compleed PDAG, he operaor migh resul in a ransiion from one compleed PDAG C o anoher. This operaor also resuls in a valid ransiion. To obain valid ransiions, Chickering [6, 7] inroduces he concep of validiy for an operaor on C. Before defining valid operaor, we need a concep consisen exension. A consisen exension of a PDAG P is a direced acyclic graph (DAG) on he same underlying se of edges, wih he same orienaions on he direced edges of P and he same se of v-srucures [10, 37]. According o Lemma 1, all consisen exensions of a PDAG P, if hey exis, belong o a unique Markov equivalence class. Hence if he modified graph of an operaor is a PDAG and has a consisen exension, i can resul in a compleed PDAG ha corresponds o a unique Markov equivalence class. We call i he resuling compleed PDAG of he operaor. Now a valid operaor is defined as below. DEFINITION 3 (Valid operaor). An operaor on C is valid if (1) he modified graph of he operaor is a PDAG and has a consisen exension, and (2) all modified edges in he modified graph occur in he resuling compleed PDAG of he operaor. The firs condiion in Definiion 3 guaranees ha a valid operaor resuls in a compleed PDAG. The second condiion guaranees ha he valid operaor is effecive; ha is, he change brough abou by he operaor occurs in he resuling compleed PDAG. Here we noice ha he second condiion is implied by he conex in Chickering [6]. Below we briefly inroduce how o obain he resuling compleed PDAG of a valid operaor from he modified graph. Verma and Pearl [37] and Meek [26] inroduce an algorihm for finding he compleed PDAG from a paern (given skeleon and v-srucures). This mehod can be used o creae he compleed PDAG from a DAG or a PDAG. They firs undirec every edge, excep for hose edges ha paricipae in a v-srucure. Then hey choose one of he undireced edges and direc i if he corresponding direced edge is srongly proeced, as shown in Figure 1(a), (c) or (d). The algorihm erminaes when here is no undireced edge ha can be direced. Chickering [6] proposes an alernaive approach o obain he compleed PDAG of a valid operaor from is modified graph; see Example 2, Secion 2.1 of he Supplemenary Maerial [16]. The mehod includes wo seps. The firs sep generaes a consisen exension (a DAG) of he modified graph (a PDAG) using he algorihm described in Dor and Tarsi [10]. The second sep creaes a compleed PDAG corresponding o he consisen exension [5, 6]. We describe Dor and Tarsi s algorihm and Chickering s algorihms in Secion 1 of he Supplemenary Maerial [16]. The approach proposed by Chickering [5, 6] is more complicaed bu more efficien [26] han Meek s mehod described above. Hence when consrucing a

6 REVERSIBLE MCMC ON MECS OF SPARSE DAGS 1747 Markov chain, we use Chickering s approach o obain he resuling compleed PDAG of a given valid operaor from is modified graph. Wih a se of valid operaors, a Markov chain on compleed PDAGs can be consruced. Le S p be he se of all compleed PDAGs wih p verices, S be a given subse of S p. For any compleed PDAG C S, leo C be a se of valid operaors of ineres o be defined laer on C in equaion (3.2). A se of valid operaors on S is defined as (1.1) O = C S O C. Here we noice ha each operaor in O is specific o he compleed PDAG ha he operaor applies o. A Markov chain {e } on S based on he se O can be defined as follows. DEFINITION 4(AMarkovchain{e } on S). The Markov chain {e } deermined by a se of valid operaors O is generaed as follows: sar a an arbirary compleed PDAG, denoed as e 0 = C 0 S, and repea he following seps for = 0, 1,...: (1) A he h sep we are a a compleed PDAG e. (2) We choose an operaor o e uniformly from O e ; if he resuling compleed PDAG C +1 of o e is in S, move o C +1 and se e +1 = C +1 ; oherwise we say a e and se e +1 = e. Given he same operaor se, he Markov chain in Definiion 4 has more new ransiion saes for any compleed PDAG han hose based on he modified graphs of operaors [25, 29, 34]. This is because some valid operaors will resul in new compleed PDAGs even if heir modified graphs are no compleed PDAGs. Consequenly, he ransiions, which are generaed by hese operaors, are no conained in Markov chains based on he modified graphs. The se S is he finie sae space of chain {e }. Clearly, he sequence of compleed PDAGs {e : = 0, 1,...} in Definiion 4 is a discree-ime Markov chain [23, 28]. Le p CC be he one-sep ransiion probabiliy of {e } from C o C for any wo compleed PDAGs C and C in S. A Markov chain {e } is irreducible if i can reach any compleed PDAG saring a any sae in S. If{e } is irreducible, here exiss a unique disribuion π = (π C, C S) saisfying balance equaions (see Theorems and in [28]) (1.2) π C = C S π C p C C for all C S. An irreducible chain e is reversible if here exiss a probabiliy disribuion π such ha (1.3) π C p CC = π C p C C for all C, C S.

7 1748 Y. HE, J. JIA AND B. YU I is well known ha π is he unique saionary disribuion of he discreeime Markov chain {e } if i is finie, reversible, and irreducible; see Lemma in [28]. Moreover, he saionary probabiliies π C can be calculaed efficienly if he Markov chain saisfies equaion (1.3). The properies of he Markov chain {e } given in Definiion 4 depend on he operaor se O. To implemen score-based searching in he whole se of Markov equivalence classes, Chickering [6] inroduces a se of operaors wih ypes of InserU, DeleeU, InserD, DeleeD, MakeV or ReverseD (reversing he direcion of a direced edge), subjec o some validiy condiions. Unforunaely, he Markov chain in Definiion 4 is no reversible if he se of Chickering s operaors is used. Our goal is o design a reversible Markov chain, as i makes i easier o compue he saionary disribuion, and hereby o sudy he properies of a subse of Markov equivalence classes. In Secion 2, we firs discuss he properies of an operaor se O needed o guaranee ha he Markov chain is reversible. Secion 2 also explains how o use he samples from he Markov chain o sudy properies of any given subse of Markov equivalence classes. In Secion 3 we focus on sudying ses of sparse Markov equivalence classes. Finally, in Secion 4, we repor he properies of direced edges and chain componens in sparse Markov equivalence classes wih up o one housand of verices. 2. Reversible Markov chains on Markov equivalence classes. Le S be any subse of he se S p ha conains all compleed PDAGs wih p verices, and O be a se of operaors on S defined in equaion (1.1). As in Definiion 4, we can obain a Markov chain denoed by {e }. We firs discuss four properies of O ha guaranee ha {e } is reversible and irreducible. They are validiy, disinguishabiliy, irreducibiliy and reversibiliy.we call a se of operaorsperfec if i saisfies hese four properies. Then we give he saionary disribuion of {e } when O is perfec and show how o use {e } o sudy properies of S A reversible Markov chain based on a perfec se of operaors. Le p CC be a one-sep ransiion probabiliy of {e } from C o C for any wo compleed PDAGs C and C in S. In order o formulae p CC clearly, we inroduce wo properies of O: Validiy and Disinguishabiliy. DEFINITION 5 (Validiy). Given S and any compleed PDAG C in S, aseof operaors O on S is valid if for any operaor o C (o wihou confusion below) in O C, o is valid according o Definiion 3 and he resuling compleed PDAG obained by applying o o C, which is differen from C, isalsoins. According o Definiion 5, if a se of operaors O on S is valid, we can move o a new compleed PDAG in each sep of {e } and he one-sep ransiion probabiliy of any compleed PDAG o iself is zero: (2.1) p CC = 0 for any compleed PDAG C S.

8 REVERSIBLE MCMC ON MECS OF SPARSE DAGS 1749 For a se of valid operaors O and any compleed PDAG C in S, wedefinehe resuling compleed PDAGs of he operaors in O C as he direc successors of C. For any direc successor of C, denoed by C, we obain p CC clearly as in equaion (2.2) ifo has he following propery. DEFINITION 6 (Disinguishabiliy). A se of valid operaors O on S is disinguishable if for any compleed PDAG C in S, differen operaors in O C will resul in differen compleed PDAGs. If O is disinguishable, for any direc successor of C, denoed by C,hereisa unique operaor in O C ha can ransform C o C. Thus, he number of operaors in O C is he same as he number of direc successors of C. Sampling operaors from O C uniformly generaes a uniformly random ransiion from C o is direc successors. By denoing M(O C ) as he number of operaors in O C,wehave { 1/M(OC ), C p CC = is a direc successor of C S; (2.2) 0, oherwise. We inroduce his propery because i makes compuaion of p CC efficien: if O is disinguishable, we know p CC righ away from M(O C ). In order o make sure he Markov chain {e } is irreducible and reversible, we inroduce wo more properies of O: irreducibiliy and reversibiliy. DEFINITION 7 (Irreducibiliy). A se of operaors O on S is irreducible if for any wo compleed PDAGs C, C S, here exiss a sequence of operaors in O such ha we can obain C from C by applying hese operaors sequenially. If O is irreducible, saring a any compleed PDAG in S, wehaveposiive probabiliy o reach any oher compleed PDAG via a sequence of operaors in O. Thus, he Markov chain {e } is irreducible. DEFINITION 8 (Reversibiliy). A se of operaors O on S is reversible if for any compleed PDAG C S and any operaor o O C wih C being he resuling compleed PDAG of o, here is an operaor o O C such ha C is he resuling compleed PDAG of o. If he se of operaors O on S is valid, disinguishable and reversible, for any pair of compleed PDAGs C, C S, C is also a direc successor of C if C is a direc successor of C.ForanyC S and any of is direc successors C,wehave (2.3) (2.4) p CC = 1/M(O C ) and p C C = 1/M(O C ). Le T = C S M(O C ), and define a probabiliy disribuion as π C = M(O C )/T.

9 1750 Y. HE, J. JIA AND B. YU Clearly, equaion (1.3) holds for π C in equaion (2.4) ifo is valid, disinguishable and reversible. π C is he unique saionary disribuion of {e } if i is also irreducible [1, 23, 28]. In he following proposiion, we summarize our resuls abou he Markov chain {e } on S, and give is saionary disribuion. PROPOSITION 1 (Saionary disribuion of {e }). Le S be any given se of compleed PDAGs. The se of operaors is defined as O = C S O C where O C is a se of operaors on C for any C in S. Le M(O C ) be he number of operaors in O C. For he Markov chain {e } on S generaed according o Definiion 4, if O is perfec, ha is, he properies validiy, disinguishabiliy, reversibiliy and irreducibiliy hold for O, hen: (1) he Markov chain {e } is irreducible and reversible; (2) he disribuion π C in equaion (2.4) is he unique saionary disribuion of {e } and π C M(O C ). The challenge is o consruc a concree perfec se of operaors. In Secion 3,we carry ou such a consrucion for a se of Markov equivalence classes wih sparsiy consrains and provide algorihms o obain a reversible Markov chain. We now show ha a reversible Markov chain can be used o compue ineresing properies of a compleed PDAG se S Esimaing he properies of S by a perfec Markov chain. ForanyC S, le f(c) be a real funcion describing any propery of ineres of C, and he random variable u be uniformly disribued on S. In order o undersand he propery of ineres, we compue he disribuion of f(u). Le s consider one example in he lieraure. The proporion of Markov equivalence classes of size one (equivalenly, compleed PDAGs ha are direced) in S p is sudied in he lieraure [14, 15, 29]. For his purpose, we can define f(u)as he size of Markov equivalence classes represened by u and obain he proporion by compuing he probabiliy of {f(u)= 1}. Le A be any subse of R, he probabiliy of {f(u) A} is (2.5) P ( f(u) A ) = {C : f(c) A,C S} S = C S I {f(c) A}, S where S is he number of elemens in he se S and I is an indicaor funcion. Le {e } =1,...,N be a realizaion of Markov chain {e } on S based on a perfec operaor se O according o Definiion 4 and M = M(O e ).Leπ(e ) be he saionary probabiliy of Markov chain {e }. From Proposiion 1,wehaveπ(e ) M for = 1,...,N. We can use {e,m } =1,...,N o esimae he probabiliy of {f(u) A} by ( ) N=1 I {f(e ) A}M ˆP 1 N f(u) A = (2.6) N=1 M 1.

10 REVERSIBLE MCMC ON MECS OF SPARSE DAGS 1751 From he ergodic heory of Markov chains (see Theorem in [28]), we can ge Proposiion 2 direcly. PROPOSITION 2. Le S be a given se of compleed PDAGs, and assume he seofoperaorso on S is perfec. The Markov chain {e } =1,...,N is obained according o Definiion 4. Then he esimaor ˆP N ({f(u) A}) in equaion (2.6) converges o P({f(u) A}) in equaion (2.5) wih probabiliy one, ha is, (2.7) P ( ˆP N ( f(u) A ) P ( f(u) A ) as N ) = 1. Proposiion 2 shows ha he esimaor defined in equaion (2.6) is a consisen esimaor of P(f (u) A). We can sudy any given subse of Markov equivalence classes via equaion (2.6) if we can obain {e } =1,...,N and {M } =1,...,N.Wenow urn o consruc a concree perfec se of operaors for a se of compleed PDAGs wih sparsiy consrains and hen inroduce algorihms o run a reversible Markov chain. 3. A Reversible Markov chain on compleed PDAGs wih sparsiy consrains. We define a se of Markov equivalence classes Sp n wih p verices and a mos n edges as follows: (3.1) S n p ={C : C is a compleed PDAG wih p verices and n C n}, where n C is he number of edges in C. Recall ha S p denoes he se of all compleed PDAGs wih p verices. Clearly, Sp n = S p when n p(p 1)/2. We now consruc a perfec se of operaors on Sp n. Noice ha our consrucions can be exended o adap o some oher ses of compleed PDAGs, say, a se of compleed PDAGS wih a given maximum degree. In Secion 3.1, we consruc he perfec se of operaors for any compleed PDAG in Sp n. In Secion 3.2, we propose algorihms and heir acceleraed version for efficienly obaining a Markov chain based on he perfec se of operaors Consrucion of a perfec se of operaors on Sp n. In order o consruc a perfec se of operaors, we need o define he se of operaors on each compleed PDAG in Sp n.lec be a compleed PDAG in Sn p. We consider six ypes of operaors on C ha were inroduced in Secion 1.2: InserU, DeleeU, InserD, DeleeD, MakeV and RemoveV. The operaors on C wih he same ype bu differen modified edges consiue a se of operaors. We inroduce six ses of operaors on C denoed by InserU C, DeleeU C, InserD C, DeleeD C, MakeV C and RemoveV C in Definiion 9. In addiion o he condiions ha guaranee validiy, for each ype of operaors, we also inroduce oher consrains o make sure ha all operaors are reversible. Firs we explain some noaion used in Definiion 9. Lex and y be any wo disinc verices in C. The neighbor se of x denoed by N x consiss of every verex

11 1752 Y. HE, J. JIA AND B. YU y wih x y in C. The common neighbor se of x and y is defined as N xy = N x N y. x is a paren of y and y is a child of x if x y occurs in C. Averexu is a common child of x and y if u is a child of boh x and y. x represens he se of all parens of x. DEFINITION 9 (Six ses of operaors on C). Le C be a compleed PDAG in Sp n and n C be he number of edges in C. We inroduce six ses of operaors on C: InserU C DeleeU C, InserD C, DeleeD C, MakeV C and RemoveV C as follows. (a) For any wo verices x,y ha are no adjacen in C, he operaor InserU x y onc is in InserU C if and only if (iu 1 )n C <n; (iu 2 ) InserU x y is valid; (iu 3 ) for any u ha is a common child of x,y in C, boh x u and y u occur in he resuling compleed PDAG of InserU x y. (b) For any undireced edge x y in C, he operaor DeleeU x y onc is in DeleeU C if and only if (du 1 ) DeleeU x y is valid. (c) For any wo verices x,y ha are no adjacen in C, he operaor InserD x y onc is in InserD C if and only if (id 1 )n C <n; (id 2 ) InserD x y is valid; (id 3 ) for any u ha is a common child of x,y in C, y u occurs in he resuling compleed PDAG of InserD x y. (d) For any direced edge x y in C, operaor DeleeD x y onc is in DeleeD C if and only if (dd 1 ) DeleeD x y is valid; (dd 2 ) for any v ha is a paren of y bu no a paren of x, direced edge v y in C occurs in he resuling compleed PDAG of DeleeD x y. (e) For any subgraph x z y in C, he operaor MakeV x z y onc is in MakeV C if and only if (mv 1 ) MakeV x z y is valid. (f) For any v-srucure x z y of C, he operaor RemoveV x z y on C is in RemoveV C if and only if (rv 1 ) x = y ; (rv 2 ) x N xy = z \{x,y}; (rv 3 ) every undireced pah beween x and y conains a verex in N xy. Muneanu and Bendou [27] discuss he consrains for he firs five ypes of operaors such ha each one can ransform one compleed PDAG o anoher. Chickering [6] inroduces he necessary and sufficien condiions such ha hese five ypes of operaors are valid. We lis he condiions inroduced by Chickering [6]in Lemma 3, Appendix A.1, and employ hem o guaranee ha he condiions iu 2, du 1, id 2, dd 1 and mv 1 in Definiion 9 hold. The se of operaors on C denoed by O C is defined as follows: O C = InserU C DeleeU C InserD C (3.2) DeleeD C MakeV C RemoveV C. Taking he union over all compleed PDAGs in Sp n, we define he se of operaors on Sp n as (3.3) O = O C, C S n p

12 REVERSIBLE MCMC ON MECS OF SPARSE DAGS 1753 where O C is he se of operaors in equaion (3.2). In he main resul of his paper, we show ha O in equaion (3.3) is a perfec se of operaors on S n p. THEOREM 1 (A perfec se of operaors on Sp n ). O defined in equaion (3.3) is a perfec se of operaors on Sp n. Here we noice ha iu 3, id 3 and dd 2 are key condiions in Definiion 9 o guaranee ha O is reversible. Wihou hese hree condiions, here are operaors ha are no reversible; see Example 3, Secion 2.1 in he Supplemenary Maerial [16]. We provide a proof of Theorem 1 in Appendix A.2. The preceding secion showed how o consruc a perfec se of operaors. A oy example is provided as Example 4 in Secion 2.1 of he Supplemenary Maerial [16]. Based on he perfec se of operaors we can obain a finie irreducible reversible discree-ime chain. In he nex subsecion, we provide deailed algorihms for obaining a Markov chain on Sp n and heir acceleraed version Algorihms. In his subsecion, we provide he algorihms in deail o generae a Markov chain on Sp n, defined in Definiion 4 based on he perfec se of operaors defined in (3.3). A skech of Algorihm 1 is shown below; some seps of his algorihm are furher explained in he subsequen algorihms. Sep A of Algorihm 1 consrucs he ses of operaors on compleed PDAGs in he chain {e }. I is he mos difficul sep and dominaes he ime complexiy of Algorihm 1. Sep B and Sep C can be implemened easily afer O e is obained. Sep D can be implemened via Chickering s mehod [6] ha was menioned in Secion 1.2. We will show ha he ime complexiy of obaining a Markov chain Algorihm 1: Road map o consruc a Markov chain on Sp n Inpu: p, he number of verices; n, he maximum number of edges; N, he lengh of Markov chain. Oupu: {e,m } =1,...,N,where{e } is Markov chain and M is he number of operaors in O e. Iniialize e 0 as any compleed PDAG in Sp n for 0 o N do Sep A Consruc he se of operaors O e in equaion (3.2) via Algorihm 1.1; Sep B Le M be he number of operaors in O e ; Sep C Randomly choose an operaor o uniformly from O e ; Sep D Apply operaor o o e.see +1 as he resuling compleed PDAG of o reurn {e,m } =1,...,N.

13 1754 Y. HE, J. JIA AND B. YU on S n p wih lengh N ({e } =1,...,N ) is approximae O(Np 3 ) if n is he same order of p. Forlargep, we also provide an acceleraed version ha, in some cases, can run hundreds of imes faser. The res of his secion is arranged as follows. In Secion 3.2.1, we firs inroduce he algorihms o implemen Sep A. In Secion we discuss he ime complexiy of our algorihm, and provide an acceleraion mehod o speed up Algorihm Implemenaion of Sep A in Algorihm 1. A deailed implemenaion of Sep A (o consruc O e ) is described in Algorihm 1.1. To consruc O e in Algorihm 1.1, we go hrough all possible operaors on e and choose hose saisfying he corresponding condiions in Definiion 9. The condiions in Algorihm 1.1 include: iu 1, iu 2, iu 3, du 1, id 1, id 2, id 3, dd 1, dd 2, rm 1, rv 1, rv 2 and mv 1. For each possible operaor, we check he corresponding condiions shown in Algorihm 1.1 one-by-one unil one of hem fails. Below, we inroduce how o check hese condiions. Algorihm 1.1: Consruc O e for a compleed PDAG e. Inpu: A compleed PDAG e wih p verices. Oupu:OperaorseO e. // All ses of possible modified edges of e used below, for example, Undireced-edges e, are generaed according o Definiion 9. 1 Se O e as empy se 2 for each undireced edge x y in Undireced-edges e do 3 consider operaor DeleeU x x, add i o O e if du 1 holds, 4 for each direced edge x y in Direced-edges e do 5 consider DeleeD x x, add i o O e if boh dd 1 and dd 2 hold; 6 for each v-srucure x z y in V-srucures e do 7 consider RemoveV x k x i x l, add i o O e if rv 1, rv 2 and rv 3 hold, 8 for each undireced v-srucure x z y in Undireced-v-srucures e do 9 consider MakeV x k x i x l, add i o O e if mv 1 holds, 10 if n e <n(i.e., iu 1 or id 1 holds) hen 11 for each pair (x, y) in Pairs-nonadj e do 12 consider InserU x y, add i o O e if iu 1, iu 2,andiu 3 hold; 13 consider InserD x y, add i o O e,ifid 1, id 2 and id 3 hold; 14 consider InserD x y, add i o O e if id 1, id 2 and id 3 hold. 15 reurn O e

14 REVERSIBLE MCMC ON MECS OF SPARSE DAGS 1755 The condiions iu 3, id 3 and dd 2 in Algorihm 1.1 depend on boh e and he resuling compleed PDAGs of he operaors. Inuiively, checking iu 3, id 3 or dd 2 requires ha we obain he corresponding resuling compleed PDAGs. We know ha he ime complexiy of geing a resuling compleed PDAG of e is O(pn e ) [6, 10], where n e is he number of edges in e. To avoid generaing resuling compleed PDAG, in he Supplemenary Maerial [16], we provide hree algorihms o check iu 3, id 3 and dd 2 only based on e and in an efficien manner. The oher condiions can be esed via classical graph algorihms. These ess include: (1) wheher wo verex ses are equal or no, (2) wheher a subgraph is a clique or no and (3) wheher all parially direced pahs or all undireced pahs beween wo verices conain a leas one verex in a given se. Checking he firs wo ypes of condiions is rivial and very efficien because he ses involved are small for mos compleed PDAGs in Sp n when n is of he same order of p. To check he condiions wih he hird ype, we jus need o check wheher here is a parially direced pah or undireced pah beween wo given verices no hrough any verices in he given se. We check his using a deph-firs search from he source verex. When looking for an undireced pah, we can search wihin he corresponding chain componen ha includes boh he source and he desinaion verices Time complexiy of Algorihm 1 and an acceleraed version. We now discuss he ime complexiy of Algorihm 1.Fore Sp n,lep and n be he number of verices and edges in e, respecively, k be he number of v-srucures in e, and k be he number of undireced v-srucures (subgraphs x y z wih x and z nonadjacen) in e. To consruc O e, in Sep A of Algorihm 1 (equivalenly, Algorihm 1.1), all possible operaors we need o go hrough: n deleing operaors (DeleeU and DeleeD), 3(p(p 1)/2 n ) insering operaors (InserU and InserD) when he number of edges in e is less han n, k RemoveV operaors and k MakeV operaors. There are a mos Q = 1.5p(p 1) 2n + k + k possible operaors for e. Among all condiions in Algorihm 1.1, he mos ime-consuming one, which akes ime O(p+ n ) [6], is o look for a pah via he deph-firs search for an operaor wih ype of InserD. We have ha he ime complexiy of consrucing O e in Algorihm 1.1 is O(Q (p + n )) in he wors case and he ime complexiy of Algorihm 1 is O( N =1 Q (p + n )) in he wors case, where N is he lengh of Markov chain in Algorihm 1. We know ha k and k reach he maxima (p 2)/2 floor(p/2) ceil(p/2) when e is a evenly divided complee biparie graphs [15]. Consequenly, he ime complexiy of Algorihm 1 are O(Np 4 ) in he wors case. Forunaely, when n is a few imes of p,sayn = 2p, all compleed PDAGs in Sp n are sparse and our experimens show k and k are much less han O(p 2 ) for mos compleed PDAGs in Markov chain {e } =1,...,N. Hence he ime complexiy of Algorihm 1 is approximae O(Np 3 ) on average when n is a few imes of p.

15 1756 Y. HE, J. JIA AND B. YU We can implemen Algorihm 1 efficienly when p is no large (less or around 100 in our experimens). However, when p is larger, we need large N o guaranee he esimaes reach convergence. Experimens in Secion 4 show N = 10 6 is suiable. In his case, cubic complexiy (O(Np 3 ))ofalgorihm1is unaccepable. We need o speed up he algorihms for a very large p. Noice ha in Algorihm 1, we obain an irreducible and reversible Markov chain {e } and a sequence of numbers {M } by checking all possible operaors on each e. The sequence {M } are used o compue he saionary probabiliies of {e } according o Proposiion 1. We now inroduce an acceleraed version of Algorihm 1 o generae irreducible and reversible Markov chains on Sp n. The basic idea is ha we do no check all possible operaors bu check some random samples. These random samples are hen used o esimae {M }. We firs explain some noaion used in he acceleraed version. For each compleed PDAG e,ifn e <n, O e (all) is he se of all possible operaors on e wih ypes of InserU, DeleeU, InserD, DeleeD, MakeV and RemoveV. If n e = n, he number of edges in e reaches he upper bound n, no more edges can be insered ino e.leo e ( inser) be he se of operaors obained by removing operaors wih ypes of InserU and InserD from O e (all). O e ( inser) is he se of all possible operaors on e when n e = n. We can obain O e (all) and O e ( inser) easily via all possible modified edges inroduced in Algorihm 1.1. The acceleraed version of Algorihm 1 is shown in Algorihm 2. In Algorihm 2, O e (eiher O e (all) or O e ( inser) ) is he se of all possible operaors on e, α (0, 1] is an acceleraion parameer ha deermines how many operaors in O e are checked, O e (check) is a se of checked operaors ha are randomly sampled wihou replacemen from O e and Õ e is he se of all perfec operaors in O e (check).whenα = 1, Õ e = O e and Algorihm 2 becomes back o Algorihm 1. In Algorihm 2, because he operaors in Õ e are i.i.d. sampled from O e in Sep A and operaor o is chosen uniformly from Õ e in Sep C, clearly, o is also chosen uniformly from O e. We have ha he following Corollary 1 holds according o Proposiion 1. COROLLARY 1 (Saionary disribuion of {e } on Sp n). Le Sn p, defined in equaion (3.1), be he se of compleed PDAGs wih p verices and maximum n of edges, O e, defined in equaion (3.2), be he se of operaors on e, and M be he number of operaors in O e. For he Markov chain {e } on Sp n obained via Algorihms 1 or 2, hen: (1) he Markov chain {e } is irreducible and reversible; (2) he Markov chain {e } has a unique saionary disribuion π and π(e ) M.

16 REVERSIBLE MCMC ON MECS OF SPARSE DAGS 1757 Algorihm 2: An acceleraed version of Algorihm 1. Inpu: α (0, 1]: an acceleraion parameer; p, n and N, he same as inpu in Algorihm 1 Oupu: {e, ˆM } =1,...,N,where ˆM is an esimaion of M = O e 1 Iniialize e 0 as any compleed PDAG in Sp n 2 for 0 o N do 3 Sep A : 4 if n e <nhen 5 Se O e = O e (all) 6 7 else Se O e = O ( inser) e 14 8 Se m = O e 9 Randomly sample [αm ] operaors wihou replacemen from O e o generae a se O e (check),where[αm ] is he ineger closes o αm. 10 Check all operaors in O e (check), and choose perfec operaors from i o consruc a se of operaors Õ e. 11 Se m (Õ) = Õ e.ifm (Õ) = 0, go o line end 13 Sep B : m Le ˆM = m (Õ) [αm ], 15 end 16 Sep C : 17 Randomly choose an operaor o uniformly from Õ e. 18 end 19 Sep D: 20 Apply operaor o o e.see +1 as he resuling compleed PDAG of o. 21 end 22 reurn {e, ˆM } =1,...,N. In Algorihm 2, we provide an esimae of M insead of calculaing i exacly in Algorihm 1. Le O e =m, O e (check) =[αm ] and Õ e =m (Õ). Clearly, he raio m (Õ) /[αm ] is an unbiased esimaor of he populaion proporion M /m via sampling wihou replacemen. We can esimae M = O e in Sep B as (3.4) m ˆM (Õ) = m [αm ].

17 1758 Y. HE, J. JIA AND B. YU We have ha when [αm ] is large, he esimaor ˆM has an approximae normal disribuion wih mean equal o M = O e. Le he random variable u be uniformly disribued on Sp n, f(u)be a real funcion describing a propery of ineres of u and A be a subse of R. By replacing M wih ˆM in equaion (2.6), we esimae P N ({f(u) A}) via {e, ˆM } =1,...,N as follows: (3.5) ˆP N ( f(u) A ) = N=1 I {f(e ) A} N=1 where P N (f (u) A) is defined in equaion (2.5). In he acceleraed version, only 100α% of all possible operaors on e are checked. In Secion 4, our experimens on S show ha he acceleraed version can speed up he approach nearly α 1 imes, and ha equaion (3.5) provides almos he same resuls as equaion (2.6) inwhich{e,m } =1,...,N from Algorihm 1 are used. Roughly speaking, if we se α = 1/p, he ime complexiy of our acceleraed version can reduce o O(Np 2 ). 4. Experimens. In his secion, we conduc experimens o illusrae he reversible Markov chains proposed in his paper and heir applicaions for sudying Markov equivalence classes. The main poins obained from hese experimens are as follows: (1) For S p wih small p, he esimaions of our proposed are very close o rue values. For Sp n wih large p (up o 1000), he acceleraed version of our proposed approach is also very efficien, and he esimaions in equaions (2.6) and(3.5) converge quickly as he lengh of Markov chain increases. (2) For compleed PDAGs in Sp n wih sparsiy consrains (n is a small muliple of p), we see ha (i) mos edges are direced, (ii) he sizes of maximum chain componens (measured by he number of verices) are very small (around en) even for large p (around 1000) and (iii) he number of chain componens grows approximaely linearly wih p. As we know, under he assumpion ha here are no laen or selecion variables presen, causal inference based on observaional daa will give a compleed PDAG. Inervenions are needed o infer he direcions of he undireced edges in he compleed PDAG. Our resuls show ha if he underlying compleed PDAG is sparse, in he model space of Markov equivalence classes, mos graphs have few undireced edges and small chain componens. They give hope for learning causal relaionships via observaional daa and for inferring he direcions of he undireced edges via inervenions. In Secion 4.1, we evaluae our mehods by comparing he size disribuions of Markov equivalence classes in S p wih small p o rue disribuions (p = 3, 4) or Gillispie s resuls (p = 6) [15]. In Secion 4.2, we repor he proporion of direced ˆM 1 ˆM 1,

18 REVERSIBLE MCMC ON MECS OF SPARSE DAGS 1759 edges and he properies of chain componens of Markov equivalence classes under sparsiy consrains. In Secion 4.3, we show experimenally ha Algorihm 2 is much faser han Algorihm 1, and ha he difference in he esimaes obained is small. Finally, we sudy he asympoic properies of our proposed esimaors in Secion Size disribuions of Markov equivalence classes in S p for small p. We consider size disribuions of compleed PDAGs in S p for p = 3, 4 and 6, respecively. There are 11 Markov equivalence classes in S 3, and 185 Markov equivalence classes in S 4. Here we can ge he rue size disribuions for S 3 and S 4 by lising all he Markov equivalence classes and calculaing he size of each explicily. Gillespie and Perlman calculae he rue size probabiliies for S 6 by lising all classes; hese are denoed as GP-values. We esimae he size probabiliies via equaion (2.6) wih he Markov chains from Algorihm 1. We ran en independen Markov chains using Algorihm 1 o calculae he mean and sandard deviaion of each esimae. The resuls are shown in Table 1, wheren is he sample size (lengh of Markov chain). We can see ha he means are very close o rue values or GP-values, and he sandard deviaions are also very small. We implemened our proposed mehod (Algorihm 1, he version wihou acceleraion) in Pyhon, and ran i on a compuer wih a 2.6 GHZ processor. In Table 1, T is he ime used o esimae he size disribuion for S 3, S 4 or S 6. These resuls were obained wihin a mos ens of seconds. In comparison, a MCMC mehod in [30] ook more han one hour (in C++ on a 2.6 GHZ compuer) in order o ge similar esimaes of he proporions of Markov equivalence classes of size one. I is worh noing ha our esimaes are based on a single Markov chain, while he resuls in [30] arebasedon10 4 independen Markov chains wih 10 6 seps Markov equivalence classes wih sparsiy consrains. We now sudy he ses Sp n of Markov equivalence classes defined in equaion (3.1). The number of verices p is se o 100, 200, 500 or 1000, and he maximum edge consrain n is se o rp where r is he raio of n o p. For each p, we consider hree raios: 1.2, 1.5 and 3. The compleed PDAGs in Sp rp are sparse since r 3. Define he size of a chain componen as he number of verices i conains. In his secion, we repor four disribuions for compleed PDAGs in Sp rp : he disribuion of proporions of direced edges, he disribuion of he numbers of chain componens and he disribuion of he maximum size of chain componens. The resuls abou he disribuion of he numbers of v-srucures are repored in he Supplemenary Maerial [16]. In each simulaion, given p and r, a Markov chain wih lengh of 10 6 on Sp rp is generaed via Algorihm 2 o esimae he disribuions via equaion (3.5). The acceleraion parameer α is se o 0.1, 0.05, 0.01 and for p = 100, 200, 500 and 1000, respecively. In Figure 2, welve disribuions of proporions of direced edges are repored for Sp rp wih differen p and raio r. We mark he minimums, 5% quariles (solid

19 1760 Y. HE, J. JIA AND B. YU TABLE 1 Size disribuions for S p wih p = 3, 4 and 6, respecively. N is he sample size, T is he ime (seconds) used o esimae he size disribuions wih a Markov chain, GP-values are obained by Gillispie and Perlman [15] p = 3, N = 10 4, T = 2sec Size True value Mean (Sd) ( ) ( ) ( ) ( ) p = 4, N = 10 4, T = 3sec Size True value Mean (Sd) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) p = 6, N = 10 5, T = 60 sec Size GP-value Mean (Sd) Size GP-value Mean (Sd) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) circles below boxes), 1s quariles, medians, 3rd quariles and maximums of hese disribuions. We can see ha for a fixed p, he proporion of direced edges increases wih he number of edges in he compleed PDAG. For example, when he raio r = 1.2, he medians (red lines in boxes) of proporions are near 92%; when he raio r = 1.5, he medians are near 95%; when raio r = 3, he medians are near 98%. The disribuions of he numbers of chain componens of compleed PDAGs in Sp rp are shown in Figure 3. We plo he disribuions for Sp 1.5p in he main window and he disribuions for r = 1.2 andr = 3 in wo sub-windows. We can see ha he medians of he numbers of chain componens are close o 5, 10, 20, and 40 for compleed PDAGs in Sp 1.5p wih p = 100, 200, 500 and 1000, respecively. I seems ha here is a linear relaionship beween he number of chain compo-

20 REVERSIBLE MCMC ON MECS OF SPARSE DAGS 1761 FIG. 2. Disribuion of proporion of direced edges in compleed PDAGs in Sp rp. The lines in he boxes and he solid circles under he boxes indicae he medians and he 5% quariles, respecively. nens and he number of verices p. In he inses, similar resuls are shown in he disribuions for r = 1.2 andr = 3. The disribuions of he maximum sizes of chain componens of compleed PDAGs in Sp rp are shown in Figure 4. ForSp 1.5p in he main window, he medians of he four disribuions are approximaely 4, 5, 6 and 7 for p = 100, 200, 500 FIG. 3. Disribuions of numbers of chain componens of compleed PDAGs in Sp rp. The lines in he boxes and he solid circles above he boxes indicae he medians and he 95% quariles, respecively.

21 1762 Y. HE, J. JIA AND B. YU FIG. 4. The disribuions of he maximum sizes of chain componens of compleed PDAGs in Sp rp. The lines in he boxes and he solid circles above he boxes indicae he medians and he 95% quariles, respecively. and 1000, respecively. This shows ha he maximum size of chain componens in a compeed PDAG increases very slowly wih p. In paricular, from he 95% quariles (solid circles above boxes), we can see ha he maximum chain componens of more han 95% compleed PDAGs in Sp 1.5p have a mos 8, 9, 10 and 13 verices for p = 100, 200, 500 and 1000, respecively. This resul implies ha sizes of chain componens in mos sparse compleed PDAGs are small Comparisons beween Algorihm 1 and is acceleraed version. Inhis secion, we show experimenally ha he acceleraed version Algorihm 2 is much faser han Algorihm 1, and he difference of esimaes based on wo algorihms is small. We have esimaed four disribuions on S in Secion 4.2 via Algorihm 2. The four disribuions are he disribuion of proporions of direced edges, he disribuion of he numbers of chain componens, he disribuion of maximum size of chain componens and he disribuion of he numbers of v-srucures. To compare Algorihm 1 wih Algorihm 2, we re-esimae hese four disribuions for compleed PDAGs in S via Algorihm 1. For each disribuion, in Figure 5, we repor he esimaes obained by Algorihm 1 wih lines and he esimaes obained by Algorihm 2 wih poins in he main windows. The differences of wo esimaes are shown in he sub-windows. The op panel of Figure 5 displays he cumulaive disribuions of proporions of direced edges. The second panel of his figure displays he disribuions of he numbers of chain componens. The hird panel displays he disribuions of maximum size of chain componens. The boom panel displays he disribuion of he

22 REVERSIBLE MCMC ON MECS OF SPARSE DAGS 1763 FIG. 5. Disribuions for compleed PDAGs in S esimaed via Algorihm 1 (ploed in lines) and he acceleraed version Algorihm 2 (ploed in poins) are shown in he main windows. The differences are shown in sub-windows. Four panels (from op o boom) display disribuions of direced edges, number of chain componens, maximum size of chain componens and v-srucures, respecively. numbers of v-srucures. We can see ha he differences of hree pairs of esimaes are small. The average imes used o generae a sae of he Markov chain of compleed PDAGs in Sp 1.5p are shown in Table 2, inwhichα is he acceleraion parameer used in Algorihm 2.Ifα = 1, he Markov chain is generaed via Algorihm 1.The resuls sugges ha he acceleraed version can speed up he approach nearly α 1 imes when p = Asympoic properies of proposed esimaors. We furher illusrae he asympoic properies of proposed esimaors of sparse compleed PDAGs via simulaion sudies. We consider Sp 1.5p for p = 100, 200, 500 and 1000, respecively. Le f(u) be a discree funcion of Markov equivalence class u, whereu is a random variable disribued uniformly in Sp 1.5p.LeE(f ) be he expecaion of f(u), TABLE 2 The average ime used o generae a compleed PDAG in Sp 1.5p, where p is he number of verices, α is he acceleraion parameer, κ is he average ime (seconds) p α κ (seconds)