A Grphil Chrteriztion of Lttie Conitionl Inepenene Moels Steen A. Anersson, Dvi Mign, Mihel D. Perlmn *, n Christopher M. Triggs Inin University, University of Wshington, n University of Aukln Astrt Lttie onitionl inepenene (LCI) moels for multivrite norml t reently hve een introue for the nlysis of non-monotone missing t ptterns n of nonneste epenent liner regression moels ( seemingly unrelte regressions). It is shown here tht the lss of LCI moels oinies with sulss of the lss of grphil Mrkov moels etermine y yli igrphs (ADGs), nmely, the sulss of trnsitive ADG moels. An expliit grphtheoreti hrteriztion of those ADGs tht re Mrkov equivlent to some trnsitive ADG is otine. This hrteriztion llows one to etermine whether speifi ADG D is Mrkov equivlent to some trnsitive ADG, hene to some LCI moel, in polynomil time, without n exhustive serh of the (possily superexponentilly lrge) equivlene lss [D]. These results o not require the existene or positivity of joint ensities. 1. Introution: Grphil Mrkov Moels. The use of irete grphs to represent possile epenenies mong sttistil vriles tes k to the work of Sewll Wright (1921) in popultion genetis n hs sine generte onsierle reserh tivity in the soil, nturl, engineering, n omputtionl sienes. Both irete n unirete grphs hve foun extensive pplitions: the former s "pth igrms" for struturl eqution moels in psyhometris n eonometris, s "Byesin networks" for expert systems in rtifiil intelligene, n s "influene igrms" in opertions reserh n mngement siene, the ltter s moels of epenene for sptil stohsti proesses, imge nlysis, n ontingeny tles. The ooks y Perl (1988), Whittker [W] (1990), Spirtes, Glymour, n Sheines (1993), Ewrs (1995), Cox n Wermuth (1996), Luritzen [L] (1996), n Jensen (1996) niely esrie these evelopments. Sine 1980, prtiulr ttention hs een irete to grphil Mrkov moels etermine y onitionl inepenene reltions mong the vriles, i.e., y the Mrkov properties speifie y the grph (f. Figure 1.1). Grphil Mrkov moels Reserh supporte in prt y the U. S. Ntionl Siene Fountion n Ntionl Seurity Ageny. *Aress for orresponene: Deprtment of Sttistis, Box 354322, University of Wshington, Settle, WA 98195-4322, USA. (mihel@stt.wshington.eu) 1
etermine y yli irete grphs (ADGs), formlly efine in Setion 2, mit espeilly simple sttistil nlysis. In prtiulr, ADG moels mit onvenient reursive ftoriztions of their joint proility ensity funtions (pf) (Luritzen et l (1990)), provie n elegnt frmework for Byesin nlysis (Spiegelhlter n Luritzen (1990)), n, in expert system pplitions, llow simple usl interprettions (Luritzen n Spiegelhlter (1988)). In the multivrite norml n multinomil ses, the likelihoo funtion (LF) (i.e., oth the joint proility ensity funtion n the prmeter spe) ftorizes reursively, whih yiels expliit mximum likelihoo estimtes (MLE) n likelihoo rtio tests (LRT) - f. [W] (1990), [L] (1996), n Anersson n Perlmn [AP] (1996). The only unirete grphil (UG) moels tht provie these onvenienes re the eomposle moels, those UG moels tht hve the sme Mrkov properties s ADG moels (Wermuth n Luritzen (1993), Dwi n Luritzen (1993), Anersson, Mign, n Perlmn [AMP] (1996)). Figure 1.1. Four yli igrphs with the vertex set V {,, }. The first three grphs re Mrkov equivlent: eh speifies the single onitionl inepenene X X X revite s, while the joint pf ftors s f(,, ) = f()f( )f( ) f( )f()f( ) f( )f( )f(). The fourth grph speifies the inepenene n the ftoriztion f()f(, )f(). For these resons, ADG moels hve eome populr ross wie rnge of pplitions; see, for exmple, Luritzen n Spiegelhlter (1988), Perl (1988), Nepolitn (1990), Spiegelhlter n Luritzen (1990), Spiegelhlter et l (1993), Mign n Rftery (1994), n York et l (1995). Inee, the tive Unertinty in Artifiil Intelligene ommunity fouses muh of its effort on ADG moels. Lttie onitionl inepenene (LCI) moels were introue y [AP] (1988, 1993) uner the itionl ssumption of multivrite normlity, motivte y nlogy with the lttie struture of lne ANOVA esigns (f. Anersson (1990)). In n LCI moel, formlly efine in Setions 4 n 7, onitionl inepenene reltions mong the vriles re speifie y the intersetion properties of finite istriutive lttie (f. Figure 1.2). LCI moels shre the esirle sttistil properties of ADG moels: the LF ftors reursively, gin yieling expliit MLEs n LRTs in the multinomil n multivrite norml ses ([AP] (1993, 1995, 1995), Perlmn n Wu [PW] (1996)). 2
{,,} {,} {,} {} Ø Figure 1.2. A finite istriutive lttie, represente s ring K of susets of {,, } I (f. 7). The LCI moel L(K) speifies the onitionl inepenene {,} {,} {}, or equivlently,, n the joint pf ftors s f(,, ) = f( )f()f( ). In the multivrite norml n multinomil ses, LCI moels re preisely suite for the nlysis of non-monotone missing t ptterns ([AP] (1991), [PW] (1996)). Suppose, for exmple, tht X 1,...,X n is smple from p-vrite norml istriution with unknown men n ovrine, n tht for eh j = 1,...,n only suvetor (X j ) Kj is oserve, where, for K {1,...,p}, X K := (X i i K). The set S :=(K j j = 1,...,n) is the oserve t pttern. It is well known tht if S is monotone, i.e., totlly orere uner inlusion, then the likelihoo funtion (LF) se on the oserve t ftors reursively into prout of onitionl norml LFs, llowing expliit likelihoo nlysis y stnr liner methos (Anerson (1957)). If S is non-monotone, however, no suh ftoriztion exists. In this se, if we let K K(S) e the ring of susets of {1,...,p} generte y S n impose the prsimonious set of CI onstrints etermine y the LCI moel L(K) ( 7), then the joint LF oes ftor reursively n expliit likelihoo nlysis y stnr liner methos is gin possile. (For exmple, if p = 3 n S = {{1,2}, {1,3}, {1,2,3}}, then K = {ø, {1}, {1,2}, {1,3}, {1,2,3}} hs the form shown in Figure 1.2 n the CI onstrint speifie y L(K) is just 2 3 1.) Similrly, LCI moels re preisely suite for the nlysis of fmily of non-neste, epenent univrite regression moels ([AP] (1994)), whih inlues Zellner's (1962) well-known seemingly unrelte regressions moel s speil se. Anlogously to the missing t prolem, if the fmily {U 1,..., U p } of regression suspes is neste, then uner the ssumption of normlity the joint LF ftors reursively n stnr liner regression methos pply. No suh reursive ftoriztion exists in the non-neste se, ut the poset forme y the regression suspes {U 1,..., U p } uner inlusion etermines (see 7) ring K of susets of {1,..., p} suh tht if the CI onstrints etermine y the LCI moel L(K) re impose, then the LF ftors reursively n mits n expliit likelihoo nlysis. Furthermore, this set of CI onstrints is prsimonious (miniml) not only with respet to the lss of ll LCI moels tht llow reursive ftoriztion for the omine regression prolem ut lso with respet to the 3
lrger ( 4) lss of ll ADG moels tht llow suh reursive ftoriztion ([AP] (1996)). In this sense, LCI moels form istinguishe sulss ( 4) of the lss of ADG moels. These onsiertions hve rise the following question: wht is the ext reltion etween the lsses of LCI n ADG moels? AMP n Triggs [AMPT] (1995) showe tht if onsiertion is restrite to multivrite istriutions with positive joint ensities, then the lss of LCI moels oinies with sulss of the lss of ADG moels, nmely, the sulss etermine y ll trnsitive ADGs (see Setion 3). The first purpose of the present pper is to show tht this restrition on the ensities n e roppe; in ft, not even the existene of joint ensities nee e ssume. This result, evelope in Setions 2-4, ppers s Theorem 4.1. Next we onsier the question of whether speifi ADG D etermines sttistil moel tht oinies with some LCI moel, i.e., whether D is Mrkov equivlent to some LCI moel. One my pply stnr polynomil-time lgorithm 1 to etermine whether D is trnsitive; if so, then the nswer is yes, y Theorem 4.1. If D is not trnsitive, however, the nswer is not neessrily no, for the following reson. Wheres t most one unirete grph n e ssoite with given grphil Mrkov moel, there my e severl (often mny) ADGs tht etermine the sme Mrkov moel - see Figure 1.1 n Setion 5 for exmples. The fmily of ll ADGs with given set of verties is nturlly prtitione into Mrkov-equivlene lsses, eh lss eing ssoite with unique sttistil moel. Although D my not e trnsitive, its Mrkov-equivlene lss my ontin trnsitive ADG, in whih se D is Mrkov equivlent to some LCI moel. Thus, in orer to etermine whether speifi ADG D is Mrkov equivlent to some LCI moel, one must nswer the following more omplex question: oes the Mrkovequivlene lss [D ] ontin t lest one trnsitive ADG? This n e eie y exhustive serh of ll memers of [D] (see Setion 5), ut sine [D] n e superexponentilly lrge, exhustive serh of [D] is omputtionlly unfesile for lrge grphs. The seon min purpose of this pper is to provie omputtionlly fesile hrteriztion, se on the essentil grph D*, of those D suh tht [D] ontins t lest one trnsitive ADG n therefore is Mrkov equivlent to some LCI moel. [AMP] (1996) show tht for eh ADG D, the equivlene lss [D] n e uniquely represente y ertin Mrkov-equivlent hin grph 2 D*, the essentil grph 3 1 Suh lgorithms require t most O(n 3 ) opertions, where n is the numer of verties. 2 Chin grphs (= iyli grphs) my hve oth irete n unirete eges ut my ontin no (prtilly) irete yles; they inlue oth ADGs n UGs s speil ses. 4
ssoite with the equivlene lss. They present n expliit hrteriztion of those grphs G suh tht G = D* for some ADG D, then pply this hrteriztion to otin polynomil-time lgorithm 4 for onstruting D* from D. These results re reviewe in Setion 5 of the present pper. In Setion 6, we hrterize those essentil grphs D* tht re Mrkov equivlent to some trnsitive ADG n show tht this hrteriztion n e verifie in polynomil time. Comine with the lgorithm for onstruting D* from D, this estlishes the omputtionl fesiility of etermining whether speifi ADG moel is Mrkov equivlent to some LCI moel. Also, we present n orienttion lgorithm tht proues Mrkov-equivlent trnsitive ADG from D* if one exists. In the sttistil pplitions of LCI moels reviewe ove, the finite istriutive lttie etermining the LCI moel is presente s ring of susets. In Setion 7 we relte this representtion to the generl efinition of n LCI moel given in Setion 4. Some si efinitions, n terminology onerning grphs re summrize in the Appenix, whih the reer is invite to review first. The representtion of n LCI moel s trnsitive ADG moel ws first suggeste y Steffen Luritzen, whom we thnk for mny helpful suggestions. 2. Grphil Mrkov Moels Determine y Ayli Digrphs. The initil isussion in this setion follows Luritzen et l. (1990) n [AP] (1996). We onsier multivrite proility istriutions P on prout proility spe X (X V), where V is finite inex set n eh X is suffiiently regulr to ensure the existene of regulr onitionl proility istriutions. Suh istriution is onveniently represente y rnom vrite X := (X V) X. For ny suset A V, we efine X A := (X A). Often we revite X n X A y n A, respetively, n efine X onstnt. For three pirwise isjoint susets A, B, n C of V, we write A B C [P] if X A n X B re onitionlly inepenent given X C uner P. If A, B, n C re not isjoint, then A B C [P] is efine to men [A\(B C)] [B\(A C)] C [P]. A grphil Mrkov moel is efine y olletion of onitionl inepenenies mong the omponent rnom vrites (X V), whih olletion is represente y n yli irete grph (ADG) D (V, E) with vertex set V in the following (equivlent) wys: 3 The essentil grph ssoite with n (equivlene lss of) ADG(s) ws first introue y Verm n Perl (1990) s the omplete pttern ssoite with the ADG. 4 Chikering (1995) n Meek (1995) lso hve otine polynomil-time lgorithms for this onstrution. 5
Definition 2.1. Let D e n ADG. A proility mesure P on X is si to stisfy: (i) the lol Mrkov property (LMP) reltive to D if, for every V, (2.1) (n()\p()) p() [P]; (ii) the glol Mrkov property (GMP) reltive to D if (2.2) A B S [P] whenever S seprtes A n B in (G n(a B S) ) m ; (iii) the well-numere Mrkov property (WNMP) reltive to D if, for eh k = 2,..., n, (2.3) k ({ 1,..., k-1 }\p( k )) p( k ) [P], where n = V n 1,..., n is well-numering of the memers of V, i.e., r < s r n( s ). Sine k p( k ) n p( k ) { 1,..., k-1 }, (2.3) is equivlent to (2.4) (p( k ) { k }) { 1,..., k-1 } p( k ) [P]. Figure 2.1. An ADG D with vertex set V {,,, }. The lol n glol Mrkov properties speify the two onitionl inepenenes n,. The term well-numering ppers s never-eresing in [AP] (1993). The following result omines Propositions 4 n 5 of Luritzen et l (1990): Theorem 2.1. Let D e n ADG. For ny proility istriution P on X, GMP LMP WNMP. It follows from Theorem 2.1 tht the WNMP oes not epen on the wellnumering hosen for V. Definition 2.2. Let D e n ADG. The set M X (D) of ll proility istriutions on X tht stisfy the three equivlent Mrkov properties LMP, GMP, n WNMP reltive to D is lle the Mrkov moel etermine y the ADG D, or, simply, the ADG moel etermine y D. 6
In pplitions n itionl prmetri ssumption, suh s multivrite normlity, is often impose on ADG moels, ut we shll not o so here. Mny exmples of speifi ADG moels pper in the referenes ite in Setion 1. It is gin emphsize tht ifferent ADGs n etermine the sme Mrkov moel. In orer to relte ADG n LCI moels in Setion 4, we now introue new Mrkov-type property etermine y n ADG D (V, E). A suset A V is lle nestrl in D if A whenever V, A, n in D. The nestrl ring A(D) is efine to e the olletion of ll nestrl susets of D. Clerly, A(D) is ring of susets of V, i.e., A(D) is lose uner unions n intersetions, hene A(D) is finite istriutive lttie uner these set opertions, n, V A(D). For ny suset A V, n(a) enotes the smllest nestrl set ontining A: n(a) = { for some A}. Definition 2.3. Let D e n ADG. A proility istriution P on X is si to stisfy the lttie onitionl inepenene property (LCIP) reltive to D if, for every pir A, B A(D), (2.5) A B A B, or, equivlently, (2.6) (A\B) (B\A) A B. Definition 2.4. Let D e n ADG. The set L X (D) of ll proility istriutions on X tht stisfy the LCIP reltive to D is lle the lttie onitionl inepenene moel (LCI moel) etermine y D. Theorem 2.2. Let D e n ADG. For ny proility istriution P on X, GMP LCIP. Thus, M X (D) L X (D). Proof. For ny pir A, B A(D), the GMP will imply (2.6) provie tht (A\B) n (B\A) re seprte y A B in (D n((a\b) (B\A) (A B)) ) m = (D n(a B) ) m = (D A B ) m ; the seon equlity follows sine A B A(D). To estlish this seprtion, onsier pir A\B n B\A suh tht there exists pth { 1,..., n } in (D A B ) m. Then there must exist n jent pir k, k+1 suh tht k A \B n k+1 B. By the efinition of the morl grph, this jeny ours iff either (i) k k+1 D A B, (ii) k k+1 D A B, or (iii) k n k+1 in D A B for some D A B. In se (i), k B 7
sine B is nestrl, hene k A B ; similrly, k+1 A B in se (ii). In se (iii), either A or B, implying tht k+1 A B or k A B, respetively. In ll ses, therefore, the pth etween n must pss through A B, hene A B seprtes A\B n B\A in (D A B ) m. This ompletes the proof. Remrk 2.1. For nonsingulr multivrite norml istriutions, Theorem 2.2 ws estlishe (in somewht ifferent ut equivlent form) in Theorem 5.1 of [AMPT] (1995). Theorem 2.2 ws isovere inepenently y Koster (1996), Proposition 3.2. 3. Trnsitive Ayli Digrphs. An ADG D (V, E) is si to e trnsitive ADG ( TADG) if D n D D, where,, V. Equivlently, D is trnsitive if n()\{} = p() for ll V. An ADG D (V, E) eomes prtilly orere set ( poset) (V, ) uner the prtil orering efine y if n(). Uner this prtil orering, < iff n()\{}, i.e., iff there is irete pth from to in D. Thus, D is trnsitive iff < D, in whih se the two reltions re equivlent: < D. Therefore, finite posets n TADGs re ientil mthemtil ojets. Theorem 3.1. Let D e TADG. For ny proility istriution P on X, LCIP WNMP. Thus, for TADG, GMP LMP WNMP LCIP n M X (D) = L X (D). Proof. Assume tht P stisfies the LCIP reltive to D. To verify (2.4), use the trnsitivity of D to rewrite (2.4) s (3.1) n( k ) { 1,..., k-1 } n( k )\{ k } [P]. But n( k ) n { 1,..., k-1 } re nestrl sets whose intersetion is n( k )\{ k }, so (3.1) is implie y the LCIP. The seon sttement then follows from Theorems 2.1 n 2.2. Figure 3.1. A trnsitive ADG D with vertex set V {,,, }. The moel M X (D) L X (D) speifies the single onitionl inepenene. Note tht A(D) {Ø, {}, {, }, {, }, {,, }, {,,, }} is isomorphi to the lttie L in Figure 4.1 - ompre to (4.1). 8
4. Lttie Conitionl Inepenene Moels. The generl lss of LCI moels is now efine n shown to oinie with the lss of ll grphil Mrkov moels etermine y TADGs. We refer to Dvey n Priestley (1990), Ch. 8, for review of the funmentl ulity etween finite istriutive ltties n finite posets. Let L L(, ) e finite istriutive lttie with minimum element 0. The suset J(L) of join-irreuile elements of L is efine s follows: J(L) := {j L j 0, j = k l j = k or j = l}. Then (J(L), ) eomes finite poset uner the prtil orering inherite from L: j k iff j k = j. A suset A J(L) is nestrl if j A whenever j J(L), k A, n j k. Note tht the ring A((J(L), )) of ll nestrl susets of the poset (J(L), ) is ientil to the nestrl ring A((J(L), E < )), where (J(L), E < ) is the TADG given y E < := {(j, k) J(L) J(L) j < k}. This justifies the use of the sme nottion "n( )" for posets (see (4.1)) n for ADGs. Birkhoff's Theorem (f. Dvey n Priestley (1990), 8.17) sttes tht the mpping (4.1) L A((J(L), )) l n(l) := {j J(L) j l} etermines lttie isomorphism etween the finite istriutive lttie L n the ring A((J(L), )). This implies, in prtiulr, tht for every pir l, m L, (4.2) n(l m) = n(l) n(m). When onvenient, we shll ientify n(l) with l for l L. A generl LCI moel etermine y the finite istriutive lttie L onsists of fmily of multivrite proility istriutions P on prout proility spe X (X j j J(L)) inexe y J(L), where gin eh X j is suffiiently regulr to ensure the existene of regulr onitionl proility istriutions. It is gin onvenient to represent suh istriution y rnom vrite X := (X j j J(L)) X. For ny suset J J(L), efine X J := (X j j J), revite X j n X J y j n J, respetively, n efine X onstnt. 9
Definition 4.1. Let L e finite istriutive lttie. A proility mesure P on X is si to stisfy the lttie onitionl inepenene property (LCIP) reltive to L if, for every l, m L, (4.3) n(l) n(m) n(l m) [P]; y ientifying n(l) with l, (4.3) n e expresse in the simpler form (4.4) l m l m [P]. Definition 4.2. Let L e finite istriutive lttie. The set L X (L) of ll proility istriutions on X tht stisfy the LCIP reltive to L is lle the lttie onitionl inepenene (LCI) moel etermine y L. Ø Figure 4.1. A finite istriutive lttie L. The LCI moel L X (L) speifies the onitionl inepenene. Here J(L) = {,,, } n the TADG (J(L), E < ) is isomorphi to the TADG D in Figure 3.1. Theorem 4.1. The lss of LCI moels oinies with the lss of TADG moels. Proof. For generl LCI moel L X (L), where X = (X j j J(L)), we hve (4.5) L X (L) = L X ((J(L), E < )) = M X ((J(L), E < )). The first equlity is estlishe y pplying Definition 2.3 to L X ((J(L), E < )), then invoking the ientity of A((J(L), E < )) n A((J(L), )) n the isomorphism (4.1); n y pplying Definition 4.1 (using (4.3)) to L X (L), then invoking (4.2). The seon equlity follows immeitely from Theorem 3.1. Thus, the LCI moel L X (L) n e expresse s the TADG moel M X ((J(L), E < )). Conversely, onsier TADG moel M X (D), where D (V, E) is TADG n where X = (X V). Then (4.6) M X (D) = L X (D) = L X (A(D)); 10
the first equlity follows from Theorem 3.1, while the seon follows from Definitions 2.3 n 4.1 (using (4.4)). Thus, M X (D) n e expresse s the LCI moel L X (A(D)). Remrk 4.1. In generl J(L) is smller tht L, so in (4.5), the TADG representtion is more eonomil thn the LCI representtion. Remrk 4.2. By Theorem 4.1, LCI moels inherit ll properties of ADG moels, in prtiulr the reursive ensity ftoriztion tht, for multivrite norml or multinomil istriutions, llows simple n expliit (non-itertive) sttistil nlysis. As note erlier, however, LCI moels omprise istinguishe sulss of ADG moels. Besies the fetures of LCI moels note in Setion 1, we here tht in the norml se, n LCI moel remins invrint uner the tion of sugroup (etermine y the prtiulr lttie) of the group of lok-tringulr mtries. This invrine les to n integrl representtion of lssil form for the istriution of the mximl invrint sttisti (= generlize eigenvlues) n to the istriution of the likelihoo rtio sttisti for testing one LCI moel ginst nother. See [AP] (1993, 1995) for etils. 5. Mrkov Equivlene of Ayli Digrphs; the Essentil Grph D*. As note in Setion 1, ifferent ADGs n etermine the sme grphil Mrkov moel. Definition 5.1. Two ADGs D 1 n D 2 re Mrkov equivlent on prout spe X inexe y V if M X (D 1 ) = M X (D 2 ). If D 1 n D 2 re Mrkov equivlent on every suh prout spe X, then D 1 n D 2 re lle Mrkov equivlent n we write D 1 ~ D 2. The Mrkov equivlene lss ontining D is enote y [D]. A well-known grph-theoreti riterion for the Mrkov equivlene of ADGs is stte in Theorem 5.1. This riterion ws first prove y Verm n Perl (1992), Corollry 3.2, - lso see [AMP] (1996), Theorem 2.1 - n inepenently y Fryenerg (1990), Theorem 5.6, for the more generl lss of hin grphs - lso see [AMP] (1996), Theorem 3.1. Theorem 5.1. Two ADGs re Mrkov equivlent if n only if they hve the sme skeleton n the sme immorlities (see Figure 5.1). 11
D 1 D 2 D 3 D 4 Figure 5.1: The four ADGs with the sme skeleton s D 1 n the immorlity (,, ). The ADGs D 1, D 2, n D 3 hve no other immorlities, hene re Mrkov equivlent y Theorem 5.1. The ADG D 4 hs the itionl immorlity (,, ), hene is not Mrkov equivlent to the others. Thus, [D 1 ] = {D 1, D 2, D 3 }. Sine n ADG with n verties n hve t most O(n 3 ) immorlities, Theorem 5.1 provies fesile riterion for eiing whether two given ADGs re Mrkov equivlent in polynomil time. However, it oes not iretly yiel hrteriztion of the entire equivlene lss [D] for given ADG D, hene oes not provie fesile riterion for eiing whether [D] ontins trnsitive ADG, i.e., whether D is Mrkov equivlent to some LCI moel. Consier, for exmple, the non-trnsitive ADG D 1 in Figure 5.2: oes [D 1 ] ontin trnsitive ADG? Theorem 5.1 oes not llow us to nswer this question y iret inspetion of D 1 ; inste, we must first etermine ll memers of [D 1 ] s follows, then hek eh memer for trnsitivity. Sine (,, ) is n immorlity in D 1, the rrows n re essentil in D 1, i.e., these rrows must our in every memer of [D 1 ]. The remining three eges of D 1 might e oriente in 2 3 = 8 possile wys, s shown in Figure 5.2. Of these 8 igrphs, only 5 re yli, n of these 5, only three (D 1, D 2, D 3 ) possess extly the sme immorlity s D 1, hene [D 1 ] = {D 1, D 2, D 3 }. Thus [D 1 ] oes ontin trnsitive ADG, nmely D 3. D 1 D 2 D 3 D 4 D 5 D 6 D 7 D 8 12
Figure 5.2: The 2 3 = 8 possile igrphs with the sme skeleton s D 1 n the immorlity (,, ). Of these 8, D 5, D 6, n D 7 re not yli, while D 4 n D 8 re yli ut possess the itionl immorlity (,, ), so [D 1 ] = {D 1, D 2, D 3 }. By ontrst, onsier the non-trnsitive ADG D:. Of the 2 3 = 8 ADGs with the sme skeleton s D, it is strightforwr to verify tht extly four hve one immorlity, hene re not Mrkov equivlent to D, while the remining four (inluing D itself) hve no immorlities n thus omprise the equivlene lss [D ]. Furthermore, no memer of [D] is trnsitive, hene the ADG moel etermine y D is not Mrkov equivlent to ny LCI moel. Sine the numer of possile orienttions of ll rrows tht o not prtiipte in ny immorlity of n ADG D grows exponentilly with the numer of suh rrows, hene super-exponentilly with the numer of verties, etermintion of the equivlene lss [D] y exhustive enumertion of possiilities rpily eomes omputtionlly infesile s the size of D inreses. A loser exmintion of the ADGs in Figure 5.2 revels, however, tht the rrow ours in every memer of [D 1 ], hene is n essentil rrow of D 1 even though it is not involve in ny immorlity of D 1. H we een le to ientify ll 3 essentil rrows of D 1 iretly from D 1 itself, it woul not hve een neessry to onsier D 5 - D 8 in orer to etermine [D 1 ]. On the other hn, it ppers neessry to etermine [D 1 ] efore we n ientify the essentil rrows of D 1. Fortuntely, this is not the se. [AMP] (1996) otin omputtionlly fesile polynomil-time lgorithm for etermining ll essentil rrows of n ADG D (Theorem 5.3 elow). This is one y first introuing n hrterizing the essentil grph D* ssoite with D (see Theorem 5.2). Questions suh s the existene of trnsitive memer of [D] n e nswere y polynomil-time inspetion of D* itself ( 6), without the nee for n exhustive serh of [D]. Definition 5.2. The essentil grph D* ssoite with D is the grph D* := (D D' ~ D), i.e., D* is the smllest grph lrger thn every D' [D]. (See Appenix for efinitions.) Thus, D* is the grph with the sme skeleton s D, ut where n ege is irete in D* iff it ours s irete ege ( rrow) with the sme orienttion in every D' [D]; ll other eges of D* re unirete. (See Figure 5.3 for exmples.) The irete eges ( 13
rrows) in D* re lle the essentil rrows of D. Clerly, every rrow tht prtiiptes in n immorlity in D is essentil, ut D my ontin other essentil rrows s well, e.g., the rrow in the seon grph in Figure 5.3 n the rrows n (verify!) in the thir grph in Figure 5.3. [AMP] (1996) show tht D* is hin grph tht is itself Mrkov equivlent to D, so tht D* ontins the sme sttistil informtion s D. Their omplete hrteriztion of essentil grphs, stte here s Theorem 5.2, involves further restritions on the onfigurtions of rrows n lines ( unirete eges) tht n our in D*. Figure 5.3: Four exmples of essentil grphs D*. In the first exmple, D is the ADG D 1 of Figure 5.1. In the seon, D is the ADG D 1 of Figure 5.2. In the thir, D = D* (see Corollry 5.2). In the fourth, D is the ADG. Definition 5.3. Let G e grph. An rrow G is strongly protete in G if ours in t lest one of the following four onfigurtions s n inue sugrph of G: (): (): (): 1 (): 2 ( 1 2 ). Theorem 5.2 (Chrteriztion of D*). A grph G (V, E) is equl to D* for some ADG D if n only if G stisfies the following four onitions: (i) G is hin grph; (ii) for every hin omponent τ of G, G τ is horl; (iii) the onfigurtion oes not our s n inue sugrph of G; (iv) every rrow G is strongly protete in G. Sine oth UGs n ADGs re hin grphs, Theorem 5.2 immeitely yiels the following two orollries. Corollry 5.1. Let G e UG. Then G = D* for some ADG D if n only if G is horl. 14
Corollry 5.2. Let G e igrph. Then G = D* for some ADG D if n only if G is n ADG n every rrow of G is strongly protete in G (note tht onfigurtion () nnot our sine G hs no unirete eges). In this se G = D = D*. The following results will e pplie in Setion 6. Proposition 5.1 ( Proposition 4.2 of [AMP] (1996)). A igrph D' is yli n equivlent to the ADG D if n only if D' is otine from D* y orienting the eges of eh (horl) hin omponent (D*) τ in ny perfet ( yli n morl) wy. Proposition 5.2 ( Proposition 4.3 of [AMP] (1996)). An ADG D is Mrkov equivlent 5 to its essentil grph D*. [AMP] (1996) presente n verifie the following polynomil-time lgorithm for onstruting the essentil grph D* ssoite with n ADG D. Theorem 5.3 (the Constrution Algorithm). Define G 0 := D. For i 1, onvert every rrow G i-1 tht is not strongly protete in G i-1 into line, otining grph G i. Stop fter k steps, where k 0 is the smllest nonnegtive integer suh tht G k = G k+1. Neessrily, k E. Then G k = D*. This lgorithm proues sequene G 0,, G k of grphs suh tht D G 0 G k = G k+1. Sine oth rrows of n immorlity re strongly protete, eh G i hs the sme immorlities s D n D*. Let n = V. Beuse the etermintion of the set of rrows tht re not strongly protete in G i-1 requires t most O(n 4 ) opertions, this lgorithm requires t most O( E n 4 ) opertions, lthough it n e implemente in more effiient fshion (f. Chikering (1995), Meek (1995)). 6. A Grphil Chrteriztion of LCI Moels. The following hrteriztion theorem les to fesile ( polynomil-time) lgorithm for eiing whether n ADG D is Mrkov equivlent to some LCI moel: Theorem 6.1. Let D (V, E) e n ADG. The Mrkov-equivlene lss [D] ontins t lest one trnsitive ADG (equivlently, D is Mrkov equivlent to some LCI moel) if 5 For the efinition of Mrkov equivlene of hin grphs, of whih UGs, ADGs, n essentil grphs re speil ses, see for exmple Appenix B of Anersson et l (1995). 15
n only if none of the following five onfigurtions ours s n inue sugrph of the ssoite essentil grph D*: (α): (β): (γ): (δ): (ε): The proof of Theorem 6.1 is omplishe y mens of the following lgorithm for orienting the unirete eges of generl essentil grph D* (V, E*). The Orienttion Algorithm. For eh vertex V, efine the egree ν() of in D* s follows: ν() := {' V ' D* or ' D*}. For eh hin omponent τ T (D*) suh tht τ 2, orient the (neessrily unirete) eges of (D*) τ s follows: Step 1.1: Define 1 1 (τ) := rgmx{ν() τ}. Step 1.2: Define τ 1 τ 1 (τ) := {' τ 1 ' (D*) τ } { 1 }. Step 1.3: ' τ 1 \{ 1 }, orient the ege 1 ' s 1 '. For i 2: Step i.1: Define i i (τ) := rgmx{ν() (τ 1 τ i-1 )\{ 1,, i-1 }}. Step i.2: Define τ i τ i (τ) := {' τ i ' (D*) τ }\{ 1,, i-1 }. Step i.3: ' τ i, orient the ege i ' s i '. Terminte the proeure fter Step k, where k := min{i { 1,, i } = τ 1 τ i }. Clerly, 1,, k re istint verties in τ, 1 τ 1, n i τ 1 τ i-1 τ for 2 i k, so k τ. Ft 1. k = τ. Proof. If k < τ, then σ := τ\{ 1,, k } Ø. Sine (D*) τ is onnete, there exists σ n i { 1,, k } suh tht i (D*) τ. But this implies tht τ i τ 1 τ k = { 1, 16
, k }, ontriting the ssumption tht σ. In view of Ft 1, { 1,, k } = τ 1 τ k = τ n ll eges in (D*) τ re oriente y this proeure. Therefore, y pplying this Orienttion Algorithm to every hin omponent τ T(D*) we otin igrph D' D* with the sme skeleton s D*. If j i (D') τ, then neessrily j < i. Also, for eh i { 2,, k }, i (τ 1 τ i-1 )\{ 1,, i-1 }, so there exists j < i suh tht j i (D') τ. Therefore, for eh i { 2,, k } there exists irete pth from 1 to i in (D') τ. Ft 2. The igrph (D') τ is yli. Proof. This is immeite from the first n thir sentenes of the preeing prgrph. Ft 3. The ADG (D') τ is morl. Proof. Suppose tht n immorlity i m j ours in (D') τ. By the lst sentene of the prgrph preeing Ft 2, there exist irete pths from 1 to i n from 1 to j in (D') τ. Sine i, j < m, neither pth ontins m. Beuse i n j re not jent in (D') τ, this ontrits the ft tht (D*) τ is horl (Theorem 5.2(ii)). Ft 4. (i) ν( 1 ) ν( i ) for i 2. (ii) If the onfigurtion m i ours in (D') j τ, then ν( m ) ν( j ). Proof. (i) is ovious. For (ii), y the efinition of m it suffies to show tht j (τ 1 τ m-1 )\{ 1,, m-1 }. This is onsequene of the following three oservtions: () i j (D') τ j τ i ; () i m (D') τ i < m τ i τ 1 τ m-1 ; () m j (D') τ m < j j { 1,, m-1 }. Proposition 6.1. Let D e n ADG n let D' e the igrph otine y pplying the Orienttion Algorithm to the ssoite essentil grph D*. Then D' is yli n equivlent to D. Proof. This is n immeite onsequene of Fts 1-3 n Proposition 5.1. Proof of Theorem 6.1. If ny of the five onfigurtions (α) -(ε) our s n inue sugrph of D*, then every orienttion of the unirete eges of D* must proue 17
non-trnsitive igrph, hene every ADG in [D] is non-trnsitive. Conversely, if none of these onfigurtions our in D*, we shll show tht the equivlent ADG D' otine y pplying the Orienttion Algorithm to D* is trnsitive. Suppose to the ontrry tht D' is non-trnsitive. For the purposes of this proof only, we shll enote the essentil rrows of D' (i.e., the rrows of D*) y» n the nonessentil rrows of D' (i.e., those otine from lines of D* y the Orienttion Algorithm) y. Thus, t lest one of the following four non-trnsitive onfigurtions must our s n inue sugrph of D': (1):»» (2):» (3):» (4): Clerly (1) nnot our in D' sine this woul violte the non-ourrene of (α) in D*, while (2) nnot our in D' sine D* woul fil to stisfy onition (iii) of Theorem 5.2. Suppose tht (3) ours in D'. We ssert tht A\B = Ø, where A := {' V ' D* or»' D*}, B := {' V ' D* or»' D*}. If A\B Ø, then eh ' A\B must our in one of the following two onfigurtions s n inue sugrph of D*: (Ι):»? ' (II):»?»' where "?" inites either no ege, n unirete ege, or irete ege of unspeifie orienttion. (The ourrene of n rrow «' in (I) or (II) woul inue irete yle in D*, hene is forien.) The sene of ny ege etween ' n in (I) or (II) woul ontrit the non-ourrene in D* of (δ) n (γ), respetively, while the ourrene of ' in (I) or (II) woul ontrit onition (iii) of Theorem 5.2. The ourrene of '» in (I) or (II) woul ontrit the non-ourrene in D* of (β) n (α), respetively. Finlly, the ourrene of»' in (I) or (II) woul ontrit the nonourrene in D* of (α). Thus A\B = Ø. Beuse B\A, it follows tht ν() B > A ν(). Sine D*, {, } τ for some τ T(D*). If = 1 1 (τ) then ν() ν() y Ft 4(i), ontrition. If = i 18
i (τ) for some i 2, then there exists j < i suh tht j (D') τ. Therefore j A B, hene either j (D') τ, j (D') τ, or» j (D') τ. Sine (D') τ, the seon n thir possiilities woul violte the yliity of D', hene the first possiility must hol. By Ft 4(ii), this implies tht ν() ν(), gin ontrition. Thus (3) nnot our in D'. Lstly, suppose tht (4) ours in D'. We gin ssert tht A\B = Ø. If A\B Ø, then s ove, eh ' A\B must our in one of the following two onfigurtions s n inue sugrph of D*: (ΙΙΙ): ' (IV):?»' The sene of ny ege etween ' n in these two onfigurtions woul ontrit the non-ourrene in D* of (ε) n (δ), respetively, while the ourrene of ' woul ontrit onitions (ii) n (iii), respetively, of Theorem 5.2. The ourrene of '» in (III) n (IV) woul ontrit oth the yliity of D* n onition (iii). Finlly, the ourrene of»' in (III) woul ontrit oth yliity n onition (iii), while its ourrene in (IV) woul ontrit the non-ourrene of (β) in D*. Thus, gin A\B = Ø. Beuse B\A, gin ν() > ν(). Here, {,, } τ for some τ T(D*). Extly s ove, however, we n show tht ν() ν(), ontrition. Thus (4) nnot our in D'. This ompletes the proof of Theorem 6.1. The first essentil grph D* in Figure 5.3 oinies with onfigurtion (β), the thir ontins (α) s n inue sugrph, while the fourth oinies with (ε), hene Theorem 6.1 implies tht none of the three ssoite ADGs is Mrkov equivlent to TADG, so none is Mrkov equivlent to ny LCI moel. The seon D* ontins none of the onfigurtions (α) - (ε) s n inue sugrph, hene its ssoite ADG is Mrkov equivlent to some TADG, hene to some LCI moel. (These fts were estlishe in Setion 5 y exhustive enumertion of [D].) As note in Setion 1, the only unirete grphil (UG) moels tht shre the menle sttistil properties of ADG moels re the eomposle moels, whih n e hrterize on the one hn s those etermine y horl UGs, n on the other s those UGs tht re Mrkov equivlent to some ADG (Dwi n Luritzen (1993); [AMP] (1996), Corollry 4.5). The following orollry hrterizes those UGs tht re Mrkov equivlent to some TADG LCI moel. 19
Corollry 6.1. A UG G is Mrkov equivlent to some LCI moel if n only if G is horl n oes not ontin onfigurtion (ε): s n inue sugrph. Proof. ("only if"): By Theorem 4.1, G is Mrkov equivlent to some TADG D, hene G is horl. By the si Mrkov equivlene theorem for hin grphs (Fryenerg (1990), Theorem 5.6; [AMP] (1996), Theorem 3.1), D u = G u G. Therefore, sine D is trnsitive, G nnot ontin onfigurtion (ε) s n inue sugrph ("if"): By Corollry 5.1, G = D* for some ADG D. Thus D* is UG, hene nnot ontin onfigurtions (α) - (δ) s inue sugrphs, n oes not ontin (ε) y ssumption. By Theorem 6.1, therefore, D is Mrkov equivlent to some LCI moel, hene, y Proposition 5.2, so is D* G. Remrk 6.1. It ws note in Setion 5 tht D* n e otine from D (V, E) y n lgorithm tht n e implemente in polynomil-time in n V. Sine eh of the onfigurtions (α) - (ε) involves t most four verties, t most O(n 4 ) opertions re require to etermine whether D* ontins ny of (α) - (ε). Thus, it is possile to etermine in polynomil-time whether speifi ADG D is Mrkov equivlent to some TADG n hene to some LCI moel. If it is equivlent, then the Orienttion Algorithm proues Mrkov-equivlent TADG D' from D*, lso in polynomil time (f. Peyton et l (1993)). 7. LCI Moels Determine y Rings of Susets. In [AP] (1991, 1993, 1994, 1995, ), [AMPT] (1995), n [PW] (1996), LCI moels re efine in slightly more restritive fshion thn in Setion 4 ove. The finite istriutive lttie is presente s ring 6 K of susets of finite inex set I n the LCI moel onsists of fmily of multivrite istriutions on prout spe of the form Y (Y i i I). Suh istriution P is represente y rnom vrite Y := (Y i i I) Y. For ny suset K I, efine Y K := (Y i i K), revite Y i n Y K y i n K, respetively, n efine Y onstnt. Then the LCI moel L(K) L Y (K) is efine to e the set of ll P suh tht (7.1) L M L M [P] for every L, M K. 6 A ring of susets of I is lose uner finite intersetions n unions, n ontins I n Ø. 20
In orer to express L Y (K) s generl LCI moel L X (L), following [AP] (1993) efine, for eh K K, <K> := (K' K K' K) [K] := K\<K>. It is strightforwr to verify tht J(K) = {J K <J> J} {J K [J] Ø}. If we now efine [J(K)] := {[J] J J(K)}, then, sine J(K) is poset uner the set inlusion orering, [J(K)] is n isomorphi poset uner the inue orering efine s follows: [J'] [J] iff J' J. Figures 1.2 n 7.1 illustrte the reltions mong K, (J(K), ), n ([J(K)], ). {, } {, } [{, }] {} {} [{, }] \ / \ / {} {} () J(K): {} {, }, {} {, } () [J(K)]: {} {}, {} {} Figure 7.1. The isomorphi posets (J(K), ) n ([J(K)], ) etermine y the ring K in Figure 1.2. Note tht K n e reovere from [J(K)] y (7.2). The ring K n e reonstrute from the memers of [J(K)] y the reltion (7.2) L = ([J] J J(K), J L) A(([J(K)], )), vli for ll L K (f. [AP] (1993), Proposition 2.1), speil se of the isomorphism (4.1). Thus, eh L K is represente uniquely s isjoint union of memers of the poset ([J(K)], ); in prtiulr 7, (7.3) I = ([J] J J(K)). We n now express L Y (K) s generl LCI moel L X (L): simply set L = K n efine X := (X J J J(K)), where X J := (Y i i [J]). Then X = Y y (7.3), n the LCI onition 7 See, for exmple, Figure 7.1(), where I = {,, }. 21
(4.4) for L X (L) oinies with the LCI onition (7.1) for L Y (K). Thus the LCI moel L Y (K) is generl LCI moel L X (L) with the slight restrition tht eh omponent spe X J is itself prout spe inexe y the memers of [J]. Appenix: Grphs. Our terminology n nottion losely follows those of Luritzen et l (1990) n Fryenerg (1990), with one exeption note elow. A grph G is pir (V, E), where V is finite set of verties n E, the set of eges, is suset of E*(V) (V V)\{(, ) V}, i.e., set of orere pirs of istint verties; thus our grphs inlue no loops or multiple eges. An ege (, ) E whose opposite (, ) E is lle n unirete ege n ppers s line in our figures, wheres n ege (, ) E whose opposite (, ) E is lle irete ege n ppers s n rrow 8 :. If G ontins only unirete eges, it is n unirete grph (UG); if G ontins only irete eges it is irete grph (igrph). It shll e onvenient to write G to inite tht (, ) E ut (, ) E; in this se we sy tht the rrow ours in G. Similrly, we write G to inite tht (, ) E n (, ) E; in this se we sy tht the line ours in G. For eh vertex V, efine p G () := { V G}, the set of prents of in G. For ny suset A V, the ounry of A in G is the set G (A) := { V\A (, ) E for some A}; the losure of A in G is the set l G (A) := G (A) A. A suset A V inues the sugrph G A := (A, E A ), where E A := E (A A). The skeleton G u of grph G (V, E) is its unerlying unirete grph, i.e., G u := (V, E u ), where E u := {(, ) (, ) E or (, ) E}. Two verties, re lle jent in G if (, ) E u. Let,, n e three istint verties of G (V, E). The triple (,, ) is lle n immorlity of G if the inue sugrph G {,, } is ; tht is, if the prents n of re unmrrie ( non-jent). A grph G 2 (V 2, E 2 ) i s si to e lrger thn grph G 1 (V 1, E 1 ), enote y G 1 G 2, if V 1 V 2 n E 1 E 2. Thus, if (G 1 ) u = (G 2 ) u, then G 1 G 2 iff G 1 n G 2 iffer only in tht some irete eges (rrows) in G 1 my e onverte into unirete eges (lines) in G 2. We write G 1 G 2 if G 1 G 2 ut G 1 G 2. The union of finite olletion of sugrphs (G i (V i, E i ) i = 1,, n) of G (V, E) is 8 Our nottion iffers from Fryenerg's in this regr: he uses the nottion rther thn in his text, lthough not in his figures. 22
the sugrph G i := ( V i, E i ). Clerly, G i is the smllest sugrph lrger thn eh G i, i = 1,, n. Let, e istint verties in G (V, E). A pth π of length n 1 from to in G is sequene π { 0, 1,, n } V of istint verties suh tht suh tht 0 =, n =, n either i-1 i G or i-1 i G for every i = 1,, n. If i-1 i G for t lest one i, the pth is irete; if this is not the se, the pth is unirete. A (irete) yle is (irete) pth with the moifition tht 0 = n. An rrow G is si to lok irete yle in G if there is irete pth from to in G other thn itself. A UG G (V, E) is omplete if ll pirs of verties re jent. Trivilly, the empty grph is omplete. A suset A V is omplete if its inue sugrph G A is omplete. A suset A V is onnete in G if for every istint pir, A, there is pth from to in G A. For pirwise isjoint susets A ( ), B ( ), n S of V, A n B re seprte y S in G if ll pths from verties in A to verties in B interset S. The UG G (V, E) is horl if every yle of length n 4 possesses hor, tht is, two non-onseutive jent verties. A totl orering of V is perfet orering of G if, when eh ege of G is oriente in orne with this orering, the resulting ADG D is perfet, i.e., is yli n morl (without immorlities); D is lle perfet irete version of G. It is well-known tht UG mits perfet irete version if n only if it is horl (f. Blir n Peyton (1993)). A grph G (V, E) is lle hin grph if it oes not ontin ny irete yles. Every inue sugrph G A of G is lso hin grph. Any UG is trivilly hin grph. A hin grph tht is lso igrph is lle n yli igrph (ADG). For the reminer of the Appenix, let G (V, E) e hin grph. Then G etermines pre-orering (V, ) s follows: iff = or there exists pth from to in G. A suset A V is n nterior set if A A. For ny suset A V, n(a) is the smllest nterior set ontining A: n(a) = { V for some A}. If oth n then we write, whih ours iff = or there is n unirete pth from to in G. Fryenerg (1990) notes tht is n equivlene reltion on V; we enote the set of equivlene lsses in V y T(G). Equivlently, T(G) is the set of onnete omponents of the unirete grph otine from G y removing ll irete eges. Eh τ T(G) is lle hin omponent of G. A onnete UG hs only one hin omponent, while for n ADG, every hin omponent onsists of single vertex. We write < if there exists irete pth from to. The future of vertex V is the set φ() := { V < }. A triple (, C, ) is lle omplex in G if C is onnete suset of hin 23
omponent τ T(G) n n re two non-jent verties in G (τ) G (C). A omplex (, C, ) is lle miniml omplex in G if no proper suset C' C forms omplex (, C', ) in G. Fryenerg(1990) notes tht (, C, ) is miniml omplex in G iff G C {, } looks like the hin grph of Figure A.1. An immorlity is the speil se of miniml omplex where C = 1. C Figure A.1: A simple hin grph. Here (, C, ) is miniml omplex. The morl grph etermine y G is the unirete grph G m (V, E m ), where E m := E u [ (E*( G (τ)) τ T(G))]. Tht is, G m is G u ugmente y ll unirete eges neee to mke G (τ) omplete in G m for every hin omponent τ T (G). Equivlently, G m is otine from G u y ing line whenever (, C, ) is miniml omplex in G. Referenes Anerson, T. W. (1957). Mximum likelihoo estimtes for multivrite norml istriution when some oservtions re missing. J. Amer. Sttist. Asso. 52 200-203. Anersson, S. A. (1990). The lttie struture of orthogonl liner moels n orthogonl vrine omponent moels. Sn. J. Sttist. 17 287-319. Anersson, S. A., D. Mign, n M. D. Perlmn (1996). On the Mrkov equivlene of hin grphs, unirete grphs, n yli igrphs. Sn. J. Sttist., in press. Anersson, S. A., D. Mign, n M. D. Perlmn (1996). A hrteriztion of Mrkov equivlene lsses for yli igrphs. Ann. Sttist., in press. Anersson, S. A., D. Mign, M. D. Perlmn, n C. M. Triggs (1995). On the reltion etween onitionl inepenene moels etermine y finite istriutive ltties n y irete yli grphs. J. Sttist. Plnning n Inferene 48 25-46. 24
Anersson, S. A. n M. D. Perlmn (1988). Lttie moels for onitionl inepenene in multivrite norml istriution. Preprint, Institute of Mthemtil Sttistis, University of Copenhgen n Deprtment of Sttistis, University of Wshington. Anersson, S. A. n M. D. Perlmn (1991). Lttie-orere onitionl inepenene moels for missing t. Sttist. Proility Letters 12 465-486. Anersson, S. A. n M. D. Perlmn (1993). Lttie moels for onitionl inepenene in multivrite norml istriution. Ann. Sttist. 21 1318-1358. Anersson, S. A. n M. D. Perlmn (1994). Norml liner moels with lttie onitionl inepenene restritions. Multivrite Anlysis n Its Applitions (T. W. Anerson, K.-T. Fng, n I. Olkin, es.) IMS Leture Notes-Monogrph Series, Vol. 24, 97-110. Anersson, S. A. n M. D. Perlmn (1995). Testing lttie onitionl inepenene moels. J. Multivrite Anlysis 53 18-38. Anersson, S. A. n M. D. Perlmn (1995). Uniseness of the likelihoo rtio test for lttie onitionl inepenene moels. J. Multivrite Anlysis 53 1-17. Anersson, S. A. n M. D. Perlmn (1996). Norml liner regression moels with reursive grphil Mrkov struture. Sumitte to J. Multivrite Anlysis. Blir, J. R. S. n B. Peyton (1993). An introution to horl grphs n lique trees. In Grph Theory n Sprse Mtrix Computtion (A. George, J. R. Gilert, n J. W. H. Liu, es.), 1-29. Springer-Verlg, New York. Chikering, D. M. (1995). A trnsformtionl hrteriztion of equivlent Byesin network strutures. In Unertinty in Artifiil Intelligene: Proeeings of the Eleventh Conferene (P. Besnr, S. Hnks, es.), 87-98. Morgn Kufmnn, Sn Frniso. Cox, D. R. n Wermuth, N. (1996). Multivrite Depenenies: Moels, Anlysis, n Interprettion. Chpmn n Hll, Lonon. Dvey, B. A. n H. A. Priestley (1990). Introution to Ltties n Orer. Cmrige U. Press, Cmrige, U.K. 25
Dwi, A. P. n S. L. Luritzen (1993). Hyper Mrkov lws in the sttistil nlysis of eomposle grphil moels. Ann. Sttist. 21 1272-1317. Ewrs, D. (1995). Introution to Grphil Moeling. Springer, New York. Fryenerg, M. (1990). The hin grph Mrkov property. Sn. J. Sttist. 17 333-353. Jensen, F. V. (1996). An Introution to Byesin Networks. U. College Lonon Press, Lonon. Koster, J. A. (1996). Mrkov properties of non-reursive usl moels. Ann. Sttist., in press. Luritzen, S. L. (1996). Grphil Moels. Oxfor University Press, Oxfor. Luritzen, S. L., A. P. Dwi, B. N. Lrsen, n H.-G. Leimer (1990). Inepenene properties of irete Mrkov fiels. Networks 20 491-505. Luritzen, S. L. n D. J. Spiegelhlter (1988). Lol omputtions with proilities on grphil strutures n their pplition to expert systems (with isussion). J. Royl Sttist. So. (Series B) 50 157-224. Mign, D. n A. E. Rftery (1994). Moel seletion n ounting for moel unertinty in grphil moels using Om's winow. J. Amer. Sttist. Asso. 89 1535-1546. Meek, C. (1995). Cusl inferene n usl explntion with kgroun knowlege. In Unertinty in Artifiil Intelligene: Proeeings of the Eleventh Conferene (P. Besnr, S. Hnks, es.), 403-410. Morgn Kufmnn, Sn Frniso. Nepolitn, R. E. (1990). Proilisti Resoning in Expert Systems. Wiley, New York. Perl, J. (1988). Proilisti Resoning in Intelligent Systems: Networks of Plusile Inferene. Morgn Kufmnn, Sn Mteo, CA. Perlmn, M. D. n L. Wu (1996). Lttie onitionl inepenene moels for missing t in ontingeny tles. In preprtion. 26
Peyton, B.W., A. Pothen, n X. Yun (1993). Prtitioning horl grph into trnsitive sugrphs for prllel sprse tringulr solution. Liner Alger Appl. 192 329-354. Spiegelhlter, D. J., A. P. Dwi, S. L. Luritzen, n R. G. Cowell (1993). Byesin nlysis in expert systems (with isussion). Sttist. Siene 8 219-283. Spiegelhlter, D. J. n S. L. Luritzen (1990). Sequentil upting of onitionl proilities on irete grphil strutures. Networks 20 579-605. Spirtes, P., Glymour, C;., n Sheines, R. (1993). Custion, Preition, n Serh. Springer, New York. Verm, T. n J. Perl (1990). Equivlene n synthesis of usl moels. In Unertinty in Artifiil Intelligene: Proeeings of the Sixth Conferene (M. Henrion et l es.) 220-227. North Holln. Verm, T. n J. Perl (1992). An lgorithm for eiing if set of oserve inepenenies hs usl explntion. In Unertinty in Artifiil Intelligene: Proeeings of the Eighth Conferene (D. Duois et l es.), Sn Frniso: Morgn Kufmn, pp. 323-330. Wermuth, N. n Luritzen, N. (1983). Grphil n reursive moels for ontingeny tles. Biometrik 70 537-552. Whittker, J. L. (1990). Grphil Moels in Applie Multivrite Sttistis. Wiley, New York. Wright, S. (1921). Correltion n ustion. J. Agriulturl Reserh 20 557-585. York, J., D. Mign, I. Heuh, n R. T. Lie (1995). Estimting proportion of irth efets y oule smpling: Byesin pproh inorporting ovrites n moel unertinty. Applie Sttistis 44 227-242. Zellner, A. (1962). An effiient metho of estimting seemingly unrelte regression equitions n tests for ggregtion is. J. Amer. Sttist. Asso. 57 348-368. 27
28