Causality. Bernhard Schölkopf and Jonas Peters MPI for Intelligent Systems, Tübingen. MLSS, Tübingen 21st July 2015

Transcription

1 Causality Bernhard Schölkopf and Jonas Peters MPI for Intelligent Systems, Tübingen MLSS, Tübingen 21st July 2015

2 Charig et al.: Comparison of treatment of renal calculi by open surgery, (...), British Medical Journal, 1986

3 Charig et al.: Comparison of treatment of renal calculi by open surgery, (...), British Medical Journal, 1986

4 J. Mooij et al.: Distinguishing cause from effect using observational data: methods and benchmarks, submitted

5 Assume P(X 1,..., X 4 ) has been induced by X 1 = f 1 (X 3, N 1 ) X 2 = N 2 X 3 = f 3 (X 2, N 3 ) X 4 = f 4 (X 2, X 3, N 4 ) G 0 X 1 X 2 X 3 N i jointly independent G 0 has no cycles X 4 Functional causal model. Can the DAG be recovered from P(X 1,..., X 4 )?

6 Assume P(X 1,..., X 4 ) has been induced by X 1 = f 1 (X 3, N 1 ) X 2 = N 2 X 3 = f 3 (X 2, N 3 ) X 4 = f 4 (X 2, X 3, N 4 ) G 0 X 1 X 2 X 3 N i jointly independent G 0 has no cycles X 4 Functional causal model. Can the DAG be recovered from P(X 1,..., X 4 )? No. JP, J. Mooij, D. Janzing and B. Schölkopf: Causal Discovery with Continuous Additive Noise Models, JMLR 2014 S. Shimizu, P. Hoyer, A. Hyvärinen, A. Kerminen: A linear non-gaussian acyclic model for causal discovery. JMLR, 2006 P. Bühlmann, JP, J. Ernest: CAM: Causal add. models, high-dim. order search and penalized regr., Annals of Statistics 2014

7 Assume P(X 1,..., X 4 ) has been induced by X 1 = f 1 (X 3 ) + N 1 X 2 = N 2 X 3 = f 3 (X 2 ) + N 3 G 0 X 1 X 4 = f 4 (X 2, X 3 ) + N 4 X 2 X 3 N i N (0, σi 2 ) jointly independent X 4 G 0 has no cycles Additive noise model with Gaussian noise. Can the DAG be recovered from P(X 1,..., X 4 )? Yes iff f i nonlinear. JP, J. Mooij, D. Janzing and B. Schölkopf: Causal Discovery with Continuous Additive Noise Models, JMLR 2014 P. Bühlmann, JP, J. Ernest: CAM: Causal add. models, high-dim. order search and penalized regr., Annals of Statistics 2014 S. Shimizu, P. Hoyer, A. Hyvärinen, A. Kerminen: A linear non-gaussian acyclic model for causal discovery. JMLR, 2006

8 Consider a distribution generated by Y = f (X ) + N Y with N Y, X ind N X Y

9 Consider a distribution generated by Y = f (X ) + N Y with N Y, X ind N X Y Then, if f is nonlinear, there is no X = g(y ) + M X with M X, Y ind N X Y JP, J. Mooij, D. Janzing and B. Schölkopf: Causal Discovery with Continuous Additive Noise Models, JMLR 2014

10 Consider a distribution corresponding to Y = X 3 + N Y with N Y, X ind N X Y with X N (1, ) N Y N (0, )

11 Y X

12 Y X

13 Y X

14 Y gam(x ~ s(y))$residuals

15 Surprise (under some assumptions): 2 variables p variables JP, J. Mooij, D. Janzing and B. Schölkopf: Causal Discovery with Continuous Additive Noise Models, JMLR 2014

16 Surprise (under some assumptions): 2 variables p variables JP, J. Mooij, D. Janzing and B. Schölkopf: Causal Discovery with Continuous Additive Noise Models, JMLR 2014 Let P(X 1,..., X p ) be induced by a... conditions identif. structural equation model: X i = f i (X PAi, N i ) - additive noise model: X i = f i (X PAi ) + N i nonlin. fct. causal additive model: X i = k PA f ik (X k ) + N i nonlin. fct. i linear Gaussian model: X i = k PA β ik X k + N i linear fct.. i (results hold for Gaussian noise)

17

18

19 GAUL GAUSS the LINEAR

20 Significant IGCI LiNGaM Additive Noise PNL 70 Accuracy (%) Not significant Decision rate (%) see also D. Lopez-Paz, K. Muandet, B. Schölkopf, I. Tolstikhin: Towards a Learning Theory of Cause-Effect Inference, ICML 2015 E. Sgouritsa, D. Janzing, P. Hennig, B. Schölkopf: Inf. of Cause and Effect with Unsupervised Inverse Regr., AISTATS 2015

21 Real data: genetic perturbation experiments for yeast (Kemmeren et al., 2014) p = 6170 genes n obs = 160 wild-types n int = 1479 gene deletions (targets known)

22 Real data: genetic perturbation experiments for yeast (Kemmeren et al., 2014) p = 6170 genes n obs = 160 wild-types n int = 1479 gene deletions (targets known) true hits: 0.1% of pairs

23 Real data: genetic perturbation experiments for yeast (Kemmeren et al., 2014) p = 6170 genes n obs = 160 wild-types n int = 1479 gene deletions (targets known) true hits: 0.1% of pairs Invariant prediction method: E = {obs, int}

24 Real data: genetic perturbation experiments for yeast (Kemmeren et al., 2014) p = 6170 genes n obs = 160 wild-types n int = 1479 gene deletions (targets known) true hits: 0.1% of pairs Invariant prediction method: E = {obs, int} JP, P. Bühlmann, N. Meinshausen: Causal inference using inv. pred.: identification and conf. intervals, arxiv, D. Rothenhaeusler, C. Heinze et al.: backshift: Learning causal cyclic graphs from unknown shift interv., arxiv M. Rojas-Carulla et al.: A Causal Perspective on Domain Adaptation, arxiv

25 ACTIVITY GENE observational training data ACTIVITY GENE 5954 interventional training data (interv. on genes other than 5954 and 4710) ACTIVITY GENE 5954 most significant pair ACTIVITY GENE interventional test data point (intervention on gene 5954) ACTIVITY GENE 5954

26 ACTIVITY GENE observational training data interventional training data (interv. on genes other than 3729 and 3730) ACTIVITY GENE interventional test data point (intervention on gene 3729) ACTIVITY GENE ACTIVITY GENE ACTIVITY GENE nd most significant pair

27 ACTIVITY GENE observational training data ACTIVITY GENE 3672 interventional training data (interv. on genes other than 3672 and 1475) ACTIVITY GENE rd most significant pair ACTIVITY GENE interventional test data point (intervention on gene 3672) ACTIVITY GENE 3672

28 # STRONG INTERVENTION EFFECTS PERFECT INVARIANT HIDDEN INVARIANT PC RFCI REGRESSION (CV Lasso) GES and GIES RANDOM (99% prediction interval) # INTERVENTION PREDICTIONS

29

30 B. Watterson: It s a magical world, Andrews McMeel Publishing, 1996