Graphical Gaussian models and their groups

Transcription

1 Piotr Zwiernik TU Eindhoven (j.w. Jan Draisma, Sonja Kuhnt) Workshop on Graphical Models, Fields Institute, Toronto, 16 Apr / 23

2 Outline and references Outline: 1. Invariance of statistical models under group actions. 2. The maximal group that leaves the graphical Gaussian model invariant. 3. Finite sample breakdown points for the maximum likelihood estimator in the class of graphical Gaussian models. This links to: Composite transformation (exponential) families; equivariant decision theory. Work of Gérard Letac and Hélène Massam on Wishart distributions; and related work of Steen Andersson (and collaborators) on Lie groups for Gaussian models. Hypergraphs, clique hypergraphs, and other related combinatorial concepts. Groups and breakdowns P. Laurie Davies, Ursula Gather. 2 / 23

3 Group invariance of models Definition (Schervish): Let P 0 be a parametric family with parameter space Θ and sample space (X, B). Let G be a group of transformations on X. We say that G leaves P 0 invariant if for each g G and each θ Θ there exists θ Θ such that P θ (A) = P θ (ga) for every A B. θ is unique and denoted by θ = g θ This induces the action of G on the model: g P θ (A) := P θ (g 1 A) = P g θ (A) If there is a density f (x; θ) then f (x; θ) = f (g x; g θ). 3 / 23

4 Gaussian Transformation models X = R m, G GL m (R) acts on R m : g x := gx density: f (x; K) = (2π) m/2 (det K) 1/2 exp{ 1 2 xt K x} Θ = S + m symmetric positive definite matrices f (x; K) = f (gx; g T Kg 1 ) so G acts on S + m : g K := g T Kg 1 Definition: Let K S + m. The composite Gaussian transformation family induced by G and K: M(G, K) = {f (x; g T K g) : g G, K K}. If K = {K 0} then Gaussian transformation family. 4 / 23

5 Gaussian graphical models undirected graph G = ([m], E), [m] := {1,..., m} f (x; K), K such that K ij = 0 if (i, j) / E Parameter space: S + G = {K S+ m : K ij = 0 if (i, j) / E}. S + G S+ m so G GL m (R) acts on S + G : g K := g T K g 1. 5 / 23

6 Definition of the group G Definition: G GL m (R) such that G S + G = S + G K S + G = g T K g 1 S + G for all g G Trivial example m = 2 if G = 1 2 then G = GL 2 (R) if G = 1 2 then [ 0 G = { 0 ] [ 0, 0 ] } full graph = G = GL m (R) no edges = G = T m S m : S m =permutation matrices T m = diagonal matrices, 6 / 23

7 Example: G : S G = , G = G0 = , if K = [K ij ] S G and g 1 = [g ij ] G 0 then g T Kg 1 equals too long g 12 g 22 K 22 + g 11 g 22 K 12 g 13 g 33 K 33 + g 11 g 33 K 13 g 22 (g 12 K 22 + g 11 K 12) g 2 22 K 22 0 g 33 (g 13 K 33 + g 11 K 13) 0 g 2 33 K 33 7 / 23

8 Preorder on the set of nodes the set of maximal cliques of G: C 2 [m] define a preorder on [m] by: i j iff C C j C implies i C. 4 1 C = {{1, 2, 3}, {1, 2, 4}} , 1, 2 3, 1, 2 4 Equivalently: i j iff N(j) j N(i) i. 8 / 23

9 The poset P C equivalence relation on [m]: i j iff i j and j i. the preorder gives a partial order on [m]/ the resulting poset: P C = ([m]/, ) Recall: i j iff C C j C implies i C. C = {{1, 2}, {1, 3, 4}, {1, 3, 5}} 4 3 the Hasse diagram / 23

10 The connected component of the identity 2 Recall: 1, 3 1 2, 3 S G = 0, G = G0 = , Proposition: Let G 0 G be the closed connected component of the identity matrix I. Then g G 0 if and only if for every i, j [m] g ij 0 = j i. 10 / 23

11 The full group G 2 Recall: 1 3 G = 0 0 0, Define G as the induced graph on [m]/. 2 3 The coloring map c : [m]/ N, For our example G = 3 1,2 4 x x. Theorem: For every G G = G 0 Aut( G, c). 11 / 23

12 Some examples G 0 = T 4 G = T 4 S 4 G 0 : g 12, g 43 can be non-zero G G 0 Z 2 G 0 = T 4 G T 4 D 8 G 0 : g 13, g 31, g 21, g 23, g 41, g 43 can be nonzero G G 0 Z 2 12 / 23

13 Brief summary We have: We will defined group invariance of models (composite) transformation families identified the maximal group that leaves the graphical Gaussian model invariant define finite sample breakdown points (fsbp) proceed with our algebraic and combinatorial analysis in order to analyse fsbp 13 / 23

14 High breakdown point a random sample: x n = (x 1,..., x n ). robustness of the sample mean: x 1 = x. The finite sample breakdown point is zero. robustness of the sample median: x 1 = / med(x). The finite sample breakdown point is 1/2. Problem: Compute the finite sample breakdown point for the maximum likelihood estimator for the graphical Gaussian model. Find bounds on the finite sample breakdown points 14 / 23

15 The maximum likelihood estimator sample covariance: S xn = 1 n n i=1 (x i x) T (x i x) log-likelihood: l(k; x n ) = n 2 (log det K S x n, K ), where S, K = trace(sk) l(k; g x n ) = l(g 1 K; x n ) modulo log det(g T g), where g x n = (gx 1,..., gx n ) K xn is G-equivariant: K g xn = g K xn = g T K xn g / 23

16 Finite sample breakdown points (unbounded) pseudometric D : S + G S+ G R D(K, L) = log det(kl 1 ) G 1 = {g : det(gg T ) 1} y n,k a random sample with k data points of x n altered fsbp( K, x n, D) := 1 n min{k : sup y n,k D( K xn, K yn,k ) = } Theorem (Davies,Gather): fsbp( K, x n, D) 1 n n n xn +1 2, where xn := 1 n max {k : A [n], A = k and g G 1 s.t. gx i = x i i A}. k 16 / 23

17 The link between xn and fsbp( K, x n, D) Let xn = 1 n (n k). For A = n k let g G 1 be s.t. gx i = x i for i A and gx i x i if i [n] \ A. Then g l x i = x i for l 1 and i A. But D( K xn, K gl x n ) = D( K xn, g l K xn ) = log det( K xn g l K 1 x n (g T ) l ) = l log det(gg T ) since g G 1. So fsbp( K, x n, D) 1 n k = 1 x n. For details on the sharper bound see: Davies, Gather, The breakdown point Examples and Counterexamples, / 23

18 G 0 -orbits in the sample space orbit of x R m : O x := {gx : g G} R m G contains T m = x and ind(x) are in the same orbit O I : the subset of points in R m with support given by I [m] Lemma: If g G stabilizes every x B R m then there exists g G 0 stabilizing all x B. Theorem: G 0 -orbits are in 1 1 correspondence with the up-sets of P C : O I := O I for all I O (P C ). J=I O [m] is the unique dense orbit. 18 / 23

19 Example: {1, 2, 3} {1, 2} {1, 3} {2, 3} {1} {2} {3} For example = shows congruence of {1} and {1, 2, 3}. 19 / 23

20 The generic case Problem: compute gen := xn assuming that x n is generic: x 1,..., x n lie in the dense G 0 -orbit we remove some other measure zero subsets Example 1: x i = g i[1, 0, 0] T ; w.l.o.g. assume x 1 = [1, 0, 0] T G x1 = {g : g = 0 a 0 } 0 0 b if ab 1 then g G 1 and no other point is stabilized; hence gen = 1/n Example 2: the four-cycle G 0 = T 4 and generic points are stabilized only by I; hence gen = 0 20 / 23

21 Bounds in the generic case Theorem: If in the Hasse diagram of P C every element covers at most one element (HD is a forest) then gen = 1 n (rank(p C) 1). Consider the graph Here gen = 2/n / 23

22 Further problems Non-generic data sets Is there a closed form formula? Is there an algorithmic procedure for arbitrary data sets? G-orbits in S + G Theorem (Letac, Massam): There is exactly one G-orbit in S + G if and only if the Hasse diagram of the induced poset P C for each connected component of G is a tree (every element covers at most one element). This happens exactly when G contains no 4-cycle or 4-chain as an induced subgraph. Provide the description of G-orbits in the general case 22 / 23

23 Thank you! 23 / 23