Accepted Manuscript. Guillaume Marrelec, Habib Benali. S (08)00059-X DOI: /j.spl Reference: STAPRO 4923.

Transcription

1 Accepted Manuscrpt Condtonal ndependence between two varables gven any condtonng subset mples block dagonal covarance matrx for multvarate Gaussan dstrbutons Gullaume Marrelec Habb Benal PII: S (8)59-X DOI: 1116/spl2816 Reference: STAPRO 4923 To appear n: Statstcs and Probablty Letters Receved date: 23 Aprl 27 Revsed date: 3 July 27 Accepted date: 9 January 28 Please cte ths artcle as: Marrelec G Benal H Condtonal ndependence between two varables gven any condtonng subset mples block dagonal covarance matrx for multvarate Gaussan dstrbutons Statstcs and Probablty Letters (28) do:1116/spl2816 Ths s a PDF fle of an unedted manuscrpt that has been accepted for publcaton As a servce to our customers we are provdng ths early verson of the manuscrpt The manuscrpt wll undergo copyedtng typesettng and revew of the resultng proof before t s publshed n ts fnal form Please note that durng the producton process errors may be dscovered whch could affect the content and all legal dsclamers that apply to the ournal pertan

2 Condtonal Independence Between Two Varables Gven Any Condtonng Subset Imples Block Dagonal Covarance Matrx for Multvarate Gaussan Dstrbutons Gullaume Marrelec ab Habb Benal ab a Inserm U678 Pars F-7513 France b Unversté Perre et Mare Cure Faculté de médecne Pté-Salpêtrère Pars F-7513 France Abstract Let x = (x ) be a multvarate Gaussan varable wth covarance matrx Σ For and n we show that f the condtonal covarance between x and x gven any condtonng set \ { } s equal to zero then Σ s block dagonal and and belong to two dfferent blocks Key words: multvarate Gaussan varables condtonal ndependence block dagonal covarance matrx 1 Introducton As ponted out by Dawd (1998) the concept of condtonal ndependence s beleved to be fundamental knowledge n the process of scentfc nference For multvarate Gaussan varables condtonal ndependence s quantfed by condtonal covarance Investgaton of such coeffcents have led to a better characterzaton of nteractons between varables n partcular through the use of condtonal ndependence graphs (Whttaker 199; Laurtzen 1996; Edwards 2) Margnal correlaton coeffcents have also been examned through covarance graphs (Kauermann 1996; Edwards 2) It would be nterestng Correspondng author Address: CHU Pté-Salpêtrère 91 boulevard de l Hôptal Pars Cedex 13 France Emal address: marrelec@medusseufr (Gullaume Marrelec) Preprnt submtted to Elsever Scence 3 July 27

3 to generalze these approaches by smultaneously consderng all possble condtonal covarances for a gven par of varables For nstance consder the case of a three dmensonal Gaussan varable x = (x 1 x 2 x 3 ) wth covarance matrx Σ If Corr [x 1 x 2 x 3 ] = then Corr [x 1 x 2 ] = Corr [x 1 x 3 ] Corr [x 2 x 3 ] (Wermuth 1976; Whttaker 199) If one furthermore has Corr [x 1 x 2 ] = t drectly comes out that ether Corr [x 1 x 3 ] = or Corr [x 2 x 3 ] = In other words the followng yelds: { Corr [x1 x 2 ] = and Corr [x 1 x 2 x 3 ] = } Σ s block dagonal To our knowledge no generalzaton of such a result has been shown yet Ths paper s a frst step n ths drecton We prove a result that demonstrates how ths approach can nform us regardng the global pattern of nteracton and shed lght nto the structure of the varables 2 Man theorem Let be a fnte set and x = (x ) be a multvarate Gaussan varable ndexed on wth covarance matrx Σ Theorem 1 Let and be two elements of and further assume that x and x are condtonally ndependent gven any set of remanng varables e \ { } Cov [x x x ] = (1) Then Σ s block dagonal and and belong to two dfferent blocks Sole consderaton of margnal and/or partal covarance s not suffcent to provde ths result for there exst covarance matrces that are not block dagonal whle ncludng varables for whch Cov [x x ] = and/or Cov [ x x x \{ı}] = Ths result can be establshed by successve examnaton of condtonal ndependence constrants (see Fg 1 for a graphcal sketch of proof) Frst Corr[x x ] = and hence Σ = We also have Cov[x x x k ] = for any k \ { } Snce Σ = ths covarance coeffcent s equal to (Anderson 1958) Cov[x x x k ] = Σ kσ k Σ kk For Cov[x x x k ] to be equal to zero we must then have Σ k Σ k = e ether Σ k = or Σ k = Ths lne of reasonng beng vald for any k { } t s possble to separate \ { } nto three sets: 1 such that Σ k and Σ k = for k 1; 1 such that Σ k = and Σ k for k 1; and 1 2

4 such that Σ k = and Σ k = for k 1 Let then be = {k l} wth k 1 and l 1 Cov[x x x ] s gven by (see Eq (A1)) ab ( 1) pos (a)+pos (b) det [ Σ \{b} \{a} (Σ ) a det [Σ ] ] (Σ ) b where pos (a) stands for the poston of a n Snce k 1 and l 1 we have Σ l = Σ k = Drect calculaton then shows that Cov[x x x ] s equal to Cov[x x x ] = Σ kσ kl Σ l Σ kk Σ ll Σ 2 kk Snce we must also have Cov[x x x ] = accordng to our hypothess ths equaton leads to Σ kl = gven that Σ k and Σ l are dfferent from zero Elements of 1 (resp 1) have hence a zero margnal correlaton to both (resp ) and all elements of 1 (resp 1) We then proceed by nducton Assume that there exst 2(N + 1) subsets and n wth n = N and one set N of such that = {} and = {}; { N N N} s a partton of nonzero margnal correlatons can only be found between between n 1 and n or between N and { N N} all margnal correlatons between n 1 and n as well as between and n are dfferent from zero n 1 and n n n 1 Snce we proved that Σ = Σ l = for l 1 Σ k = for k 1 Σ kl = for (k l) 1 1 and constructed 1 and 1 so that Σ k for k 1 and Σ l for l 1 the assumpton holds for N = 1 We now assume that t also holds for a gven N 1 If N s empty then the process stops Otherwse the frst step conssts of settng = {k 1 l 1 k N l N m} wth (k n l n ) n n for n = 1 N and m N Gven the assumpton of ndependence between x and x we must have Cov[x x x ] = Ths condtonal covarance coeffcent s equal to (cf Eq (A2)) Σ k1 Σ l1 [ n=1n 1 Σ knk n+1 Σ lnl n+1 ] ΣkN mσ ln m det [Σ ] and s equal to zero f and only f Σ kn mσ ln m = snce by constructon all Σ knk n+1 and Σ lnl n+1 are dfferent from zero It s then possble to separate N nto three sets: N+1 such that Σ kn m and Σ ln m = for all m N+1; N+1 such that Σ kn m = and Σ ln m for all m N+1; and N+1 such that Σ kn m = Σ ln m = for all m N+1 It now remans to prove that we have Σ kl = for (k l) N+1 N+1 To ths am set = {k 1 l 1 k N+1 l N+1 } wth (k n l n ) n n for n = 1 N +1 Snce 3

5 x and x are ndependent we must have Cov[x x x ] = Ths quantty beng equal to (see Eq (A3)) Cov [x x x ] = Σ [ ] k 1 Σ l1 n=1n Σ knk n+1σ lnl n+1 ΣkN+1l N+1 det [Σ ] t s equal to zero f and only f Σ kn+1 l N+1 = The assumpton s therefore also vald for N + 1 The sequence ( N) s of decreasng cardnal beng a fnte set there exsts a step N for whch N s empty: the process ends there Set = {V N } and = { N } { } s hence a partton of for whch there exsts no margnal correlaton between an element of and an element of Consequently the covarance matrx of x has the followng structural form: Σ Σ thereby provng the theorem 3 Dscusson and perspectves In ths paper we consdered x = (x ) a multvarate Gaussan varable wth covarance matrx Σ For and n we showed that f the condtonal covarance between x and x gven any condtonng set \ { } was equal to zero then Σ was block dagonal and and belonged to two dfferent blocks Note that the converse of ths theorem s straghtforward Indeed f one consders that the covarance matrx Σ s block dagonal then any condtonal covarance between varables belongng to two dfferent blocks s equal to zero accordng to Eq (A1) Theorem 1 shows that for multvarate Gaussan varables there s a clear separaton between two varables x and x that are ndependent wth regard to any condtonng subset and that ths separaton also apples to all other varables whch are ether wth x or wth x Consequently ther effect can be analyzed ndependently n one block of varables or the other Interestngly ths result ncely relates two dstnct propertes of Gaussan dstrbutons The block dagonal property of the covarance matrx s clearly a global feature of Gaussan probablty dstrbutons By contrast the relatonshp of complete ndependence (e condtoned on all subsets) s rather a local descrpton and characterzaton of the nteracton structure between varables snce the defnton gves a partcular role to x and x Ths perspectve dffers from the common approach where one usually sets a level 4

6 of condtonng (margnal for covarance graphs partal for condtonal ndependence graphs) and then vares the two varables on whch correlaton s calculated In ths dual approach the defnton does not so much depend on the condtonng set than on the varables whose condtonal covarance we examne We manly focus on the ndependence pattern that can be exhbted wth a sngle par of varables and ts potental mplcatons onto the global structure We beleve that there s much to gan by analyzng varables from ths perspectve and hope to be able to provde further results along the same lnes n the near future 4 Acknowledgments We are n debt to an anonymous referee for pontng out that the result exposed here s well-known for three dmensonal Gaussan varables A Calculaton of Cov [x x x ] The condtonal covarance between and gven reads (Anderson 1958) Cov[x x x ] = Σ ab (Σ ) a [ (Σ ) 1] ab (Σ ) b Calculatng (Σ ) 1 from the adont matrx (Horn and Johnson 1999) yelds for Cov[x x x ]: Σ ab ( 1) pos (a)+pos (b) det [ Σ \{b} \{a} (Σ ) a det [Σ ] ] (Σ ) b (A1) where pos (a) stands for the poston of a n From now on we also assume that there exst 2(N +1)+1 subsets of namely n n wth n = N and N respectng the condtons detaled on page 3 Frst for N 1 set = {k 1 l 1 k N l N m} k n n and l n n for n = 1 N and m N By constructon only elements n 1 (resp 1) have nonzero margnal covarance wth (resp ) Consequently the sum n Equaton (A1) can be smplfed nto ] ( 1) pos (a)+pos (b) det [ Σ \{l 1 } \{k 1 } Σ k1 Σ l1 det [Σ ] Gven the defnton of Σ Σ \{l 1 } \{k 1 } and the determnant of the latter matrx respectvely read 5

7 wth Σ = Σ \{l 1 } \{k 1 } = Σ k1 k 1 Σ k1 k 2 Σ l1 l 1 Σ l1 l 2 Σ k1 k 2 Σ k2 k 2 Σ k2 k 3 Σ l1 l 2 Σ l2 l 2 Σ l2 l 3 Σ ln 1 l N Σ ln 1 l N 1 Σ ln 1 l N 1 Σ kn 1 k Σ N kn k Σ N kn m Σ ln 1 l N Σ ln l N Σ ln m Σ kn m Σ ln m Σ mm Σ k1 k 2 Σ k2 k 2 Σ k2 k 3 Σ l1 l 2 Σ l2 l 2 Σ l2 l 3 Σ k2 k 3 Σ k3 k 3 Σ k3 k 4 Σ ln 1 l N Σ ln 1 l N 1 Σ ln 1 l N 1 Σ kn 1 k Σ N kn k Σ N kn m Σ ln 1 l N Σ ln l N Σ ln m Σ kn m Σ ln m Σ mm det [ Σ \{l 1 } \{k 1 }] = Σk1 k 2 det ( Σ \{l 1 k 1 } \{k 1 k 2 }) Σ \{l 1 k 1 } \{k 1 k 2 } = and hence Ths leads to Σ k2 k 3 Σ l1 l 2 Σ l2 l 2 Σ l2 l 3 Σ k3 k 3 Σ k3 k 4 Σ l2 l 3 Σ l3 l 3 Σ l3 l 4 Σ ln 1 l N Σ ln 1 l N 1 Σ ln 1 l N 1 Σ kn 1 k Σ N kn k Σ N kn m Σ ln 1 l N Σ ln l N Σ ln m Σ kn m Σ ln m Σ mm det ( Σ \{l 1 k 1 } \{k 1 k 2 }) = Σl1 l 2 det ( Σ \{l 1 k 1 l 2 } \{k 1 l 1 k 2 }) det [ Σ \{l 1 } \{k 1 }] = Σk1 k 2 Σ l1 l 2 det ( Σ \{l 1 k 1 l 2 } \{k 1 l 1 k 2 }) 6

8 wth Σ \{l 1 k 1 l 2 } \{k 1 k 2 l 1 } = Σ k2 k 3 Σ k3 k 3 Σ k3 k 4 Σ l2 l 3 Σ l3 l 3 Σ l3 l 4 Σ l3 l 4 Σ k4 k 4 Σ k4 k 5 Σ ln 1 l N Σ ln 1 l N 1 Σ ln 1 l N 1 Σ kn 1 k Σ N kn k Σ N kn m Σ ln 1 l N Σ ln l N Σ ln m Σ kn m Σ ln m Σ mm whch s of the same form as Σ \{l 1 } \{k 1 } Consequently a smlar calculaton shows that det ( Σ \{l 1 k 1 l 2 } \{k 1 l 1 k 2 }) = Σk2 k 3 Σ l2 l 3 det ( Σ \{l 1 k 1 l 2 k 2 l 3 } \{k 1 l 1 k 2 l 2 k 3 }) and by nducton one can hence easly show that for all n 2 det ( Σ \{l 1 k 1 l n 1 k n 1 l n} \{k 1 l 1 k n 1 l n 1 k n}) = Σ knk n+1 Σ lnl n+1 det ( Σ \{l 1 k 1 l nk nl n+1 } \{k 1 l 1 k nl nk n+1 l n}) We hence obtan that det [ Σ \{l 1 } \{k 1 }] = det ( Σ \{l 1 k 1 k N 2 l N 2 l N 1 } \{k 1 l 1 k N 2 l N 2 k N 1 } where the matrx of the rght-hand sde s equal to Σ {kn 1 k N l nm}{l N 1 k N l N m} = Σ kn 1 k N Σ kn k N Σ kn m Σ ln 1 l N Σ ln l N Σ ln m Σ kn m Σ ln m Σ mm ) The determnant of ths matrx can be obtaned by a smlar argument as prevously developed: det ( ) Σ kn m Σ {kn 1 k N l nm}{l N 1 k N l N m} = ΣkN 1k Σ ln 1 l Σ N N ln l Σ N ln m We fnally have det [ Σ \{l 1 } \{k 1 }] = ΣkN mσ ln m Σ ln m Σ mm Σ = Σ kn 1 k N Σ kn m ln 1 l N Σ ln m Σ mm = Σ kn 1 k N Σ ln 1 l N Σ kn mσ ln m 7 n=1n 1 Σ knk n+1 Σ lnl n+1 n=1n 2 Σ knk n+1 Σ lnl n+1

9 and n concluson for = {k 1 l 1 k N l N m} we obtan for Cov [x x x ] [ ] [ ] Σ k1 n=1n 1 Σ knk n+1 ΣkN m Σ l1 n=1n 1 Σ lnl n+1 ΣlN m (A2) det [Σ ] The second case s rather smlar to the frst one Set = {k 1 l 1 k N+1 l N+1 } wth N 1 k n n and l n n for n = 1 N +1 The prevous lne of reasonng can be appled n ths case too except that Σ \{l 1 k 1 l N 1 } \{k 1 l 1 k N 1 } reads Σ {kn 1 k N l N k N+1 l N+1 }{l N 1 k N l N k N+1 l N+1 } = Σ kn 1 k N Σ kn k N Σ kn k N+1 Σ ln 1 l N Σ ln l N Σ ln l N+1 Σ kn k N+1 Σ kn+1 k N+1 Σ kn+1 l N+1 Σ ln l N+1 Σ kn+1 l N+1 Σ ln l N leadng to a determnant of Σ \{l 1 k 1 l N 1 } \{k 1 l 1 k N 1 } equal to Σ kn k N+1 Σ = Σ ln 1 l Σ kn 1 k N N ln l Σ N ln l N+1 Σ kn+1 k Σ N+1 kn+1 l N+1 Σ ln l Σ N+1 ln l N Σ kn k N+1 = Σ kn 1 k N Σ ln 1 Σ l N kn+1 k Σ N+1 kn+1 l N+1 Σ ln l Σ N+1 ln l N Σ kn+1 l N+1 = Σ kn 1 k N Σ ln 1 l N Σ kn k N+1 Σ ln l N+1 Σ ln l N = Σ kn 1 k N Σ ln 1 l N Σ kn k N+1 Σ ln l N+1 Σ kn+1 l N+1 Fnally Cov [x x x ] reads [ ] [ ] Σ k1 n=1n Σ knk n+1 Σl1 n=1n Σ lnl n+1 ΣkN+1 l N+1 (A3) det [Σ ] References Anderson T W 1958 An Introducton to Multvarate Statstcal Analyss Wley Publcatons n Statstcs John Wley and Sons New York Dawd A P 1998 Condtonal ndependence In: Kotz S Read C B Banks D L (Eds) Encyclopeda of Statstcal Scences Vol 2 Wley pp Edwards D 2 Introducton to Graphcal Modellng 2nd Edton Sprnger Texts n Statstcs Sprnger New York 8

10 Horn R A Johnson C R 1999 Matrx Analyss Cambrdge Unversty Press Kauermann G 1996 On a dualzaton of graphcal Gaussan models Scand J Statst Laurtzen S L 1996 Graphcal Models Oxford Unversty Press Oxford Wermuth N 1976 Analoges between multplcatve models n contgency tables and covarance selecton Bometrcs Whttaker J 199 Graphcal Models n Appled Multvarate Statstcs J Wley and Sons Chchester 9

11 1 (a) Fg 1 Sketch of proof From an orgnal parttonng of nto { N that for all elements of N there can be no margnal covarance wth both N and covarate wth N (gathered n N+1 ) elements that covarate wth (gathered n (gathered n N+1) (b) Last we show that elements of N+1 and (b) N N+1 N N N+1 N } (a) we We then p ) and elem must have zero margnal co