Causality in Econometrics (3)
|
|
- Wesley Manning
- 6 years ago
- Views:
Transcription
1 Graphical Causal Models References Causality in Econometrics (3) Alessio Moneta Max Planck Institute of Economics Jena 26 April 2011 GSBC Lecture Friedrich-Schiller-Universität Jena Causality in Econometrics 1/27
2 Graphical Causal Models Terminology and Representation of Statistical Dependence Causality in Econometrics 2/27
3 Sources and Motivations The graphical-models approach to causal inference was mainly developed by: Spirtes, Glymour, Scheines (2000), Causation, Prediction, and Search, 2 nd edition. Pearl (2000), Causality: Models, Reasoning, and Inference. Forerunners: J.S. Mill C. Spearman T. Haavelmo, H. Wold, H. Simon H. Reichenbach, P. Suppes Causality in Econometrics 3/27
4 Sources and Motivations Ideas: Use of probability + diagrams to represent associations in the data Use of graph-theory to represent and analyze causal relations This permits, in particular: addressing the symmetry problem, typical of probabilistic approaches representation of structures where interventions are possible Formalization of the relationship between probabilistic and causal representation Emphasis on inference, agnosticism about causal ontology. But: many points of contact with probabilistic approach (Reichenbach) manipulability theory (Woodward). Causality in Econometrics 4/27
5 Formal preliminaries Graph: < V, M, E > set V of vertices (or nodes) to represent variables. set M of marks as >, (or EM empty mark), o, to represent directions of causal influences. set E of edges, which are pairs of the form {[V 1, M 1 ], [V 2, M 2 ]}, to represent causal relationships. V 1 V2 V 3 G: < {V 1, V 2, V 3 }, {EM, >}, {{[V 1, EM], [V 2, >]}, {[V 1, EM], [V 3, EM]}, {[V 3, EM], [V 2, >]}} > Causality in Econometrics 5/27
6 Formal preliminaries Undirected graph: graph in which the set of marks M = {EM} Directed graph: graph in which the set of marks M = {EM, >} and for each edge in E the marks are are always: EM, > Directed edges: A B ( {[A, EM], [B, >]}) A : parent, B : child (descendant). Causality in Econometrics 6/27
7 Formal preliminaries Path: undirected path: a sequence of vertices A,..., B such that for every pair of vertices X, Y adjacent (in the sequence) there is a connecting edge {[X, M 1 ][Y, M 2 ]}. directed path: a sequence of vertices A,..., B such that for every pair of vertices X, Y adjacent (in the sequence) there is a connecting edge {[X, EM][Y, >]}. acyclic path: path that contains no vertex more than once, otherwise it is cyclic. Causality in Econometrics 7/27
8 Example Graphical Causal Models References Introduction (In)dependence Probabilistic Inference V 1 V2 V4 V5 V 3 Directed paths: < V 1, V 2, V 4, V 5 >; < V 3, V 2, V 4, V 5 >; < V 2, V 4, V 5 >, etc. Undirected paths: < V 1, V 3, V 2, V 4, V 5 >; < V 1, V 2, V 3 >, etc. Undirected cyclic path: < V 1, V 2, V 3, V 1 > No directed cyclic paths. Causality in Econometrics 8/27
9 More terminology Collider: vertex V such that A V B Unshielded collider: vertex V such that A V B and A and B are not adjacent ( connected by edge) in the graph Complete graph: graph in which every pair of vertices are adjacent Directed Acyclic Graph (DAG): directed graph that contains no directed cyclic paths Directed Cyclic Graph (DCG): directed graph that contains directed cyclic paths Causality in Econometrics 9/27
10 Graphs and probabilistic dependence First use of graphs: representation of probabilistic dependence and independence Nodes: random variables (discrete or continuous). Edges: probabilistic dependence. Bayesian networks (Pearl 1985). Causality in Econometrics 10/27
11 Conditional Independence If X, Y, Z are random variables, we say that X is conditionally independent of Y given Z, and write X Y Z (1) if for discrete variables: P(X = x, Y = y Z = z) = P(X = x Z = z)p(y = y Z = z) for continuous variables: f XY Z (x, y z) = f X Z (x z)f Y Z (y z) We can also write (simplifying the notation): X Y Z f (x, y, z)f (z) = f (x, z)f (y, z) Causality in Econometrics 11/27
12 Conditional independence Some equalities: X Y Z f (x, y z) = f (x z)f (y z) X Y Z f (x, y, z)f (z) = f (x, z)f (y, z) X Y Z f (x y, z) = f (x z) X Y Z f (x, z y) = f (x z)f (z y) X Y Z f (x, y, z) = f (x z)f (y, z) Note: f (x, y z) = f (x, y, z)/f (z) Causality in Econometrics 12/27
13 Conditional independence It holds also: X Y Z Y X Z (symmetry) If Z is empty (trivial) X Y: X is independent of Y. Other properties: X YW Z = X Y Z (decomposition) X YW Z = X Y ZW (weak union) See Pearl 2000:11 Causality in Econometrics 13/27
14 Interpretations of C.I. Useful interpretations of C.I. X Y Z: once we know Z, learning the value of Y does not provide additional information about X. once we know Z, reading X is irrelevant for reading Y. once we observe realizations of Z, observing realizations of Y is irrelevant for predicting the frequent realizations of X. Causality in Econometrics 14/27
15 Independence and uncorrelatedness Important to distinguish between (conditional) independence and (conditional or partial) correlation. Recall: Variance of X: σ 2 X := E[(X E(X))2 ] Covariance between X and Y: σ XY := E[(X E(X))(Y E(Y))] Correlation coefficient (Pearson): ρ XY := σ XY σ X σ Y Linear regression coefficient: r XY := σ XY σ 2 Y = ρ XY σ X σ Y This suggest that correlation is a measure of linear dependence Notice: σ XY = σ YX and ρ XY = ρ YX but r XY = r YX Causality in Econometrics 15/27
16 Independence and uncorrelatedness Recall: Partial correlation between X and Y given Z ρ XY.Z = ρ XY ρ YZ ρ XZ 1 ρ 2 YZ 1 ρ 2 XZ Conditional independence X Y Z: f XY Z (x, y z) = f X Z (x z)f Y Z (y z) It holds: X Y = ρ XY = 0 X Y Z = ρ XY.Z = 0 and (of course): ρ XY = 0 = X / Y ρ XY.Z = 0 = X / Y Z Causality in Econometrics 16/27
17 Independence and uncorrelatedness In general: ρ XY = 0 = X Y ρ XY.Z = 0 = X Y Z However, if the joint distribution F(XYZ) is normal: ρ XY = 0 = X Y ρ XY.Z = 0 = X Y Z Causality in Econometrics 17/27
18 Population and sample Notice also the difference between population parameters and sample statistics: ρ XY = σ XY σ X σ Y ˆρ XY = n k=1 (X k X)(Y k Ȳ) n k=1 (X k X) 2 n k=1 (Y k Ȳ) 2 r YX = σ XY σ 2 X ˆr YX = n k=1 (X k X)(Y k Ȳ) n k=1 (X k X) 2 ˆβ OLS = (X X) 1 XY, for vectors of data X (X 1,..., X n ), Y (Y 1,..., Y n ) and where X = n 1 ΣX i. Notice that when X = 0 and Ȳ = 0, ˆr YX = ˆβ OLS. Causality in Econometrics 18/27
19 Other concepts related to independence If, given the r.v. X and Y, the moments E(X k ) < and E(Y m ) <, it turns out that X Y iff E(X k Y m ) = E(X k )E(Y m ), for all k, m = 1, 2,... X and Y are (k, m)-order dependent iff E(X k Y m ) = E(X k )E(Y m ), for any k, m = 1, 2,... (1-1)-order linear dependence: E(XY) = E(X)E(Y) (1-1)-order independence: E(XY) = E(X)E(Y) E{[X E(X)][Y E(Y)]} = 0 σ XY = 0 ρ XY = 0 Orthogonality E(XY) = 0 Note: 1 if X and Y are uncorrelated (ρ XY = 0), this is equivalent to say that their mean deviations are orthogonal (if X and Y are centered, subtracting their mean, they become orthogonal). 2 if X and Y are orthogonal, ρ XY = 0 only if E(X) = 0 or E(Y) = 0 Causality in Econometrics 19/27
20 Other concepts related to independence r-th order independence E(Y r X = x) = 0 for all x R X In summary: independence = 1 st -order independence = non-correlation orthogonality mean-subtracted variables non-correlation = dependencies!) independence (there could be non-liner (cfr. Spanos 1999: ) Causality in Econometrics 20/27
21 Statistical model Importance of defining a statistical model. Typical statistical model for continuous set of n random variables X Probability model: defines a family of density functions f (x; θ) defined over the range of values of X; Sampling model: X ((T n) matrix of data) is a random sample. (cfr. Spanos 1999: 33) Causality in Econometrics 21/27
22 The Markov Condition The Markov condition permits the representation of probabilistic dependence through a DAG. In particular, it imposes a relationship between the Bayesian network (DAG in which nodes are random variables) and the probabilistic structure. A directed acyclic graph G over V (set of vertices) and a probability distribution P(V) satisfy the Markov condition iff for every W V, W V\(Descendants(W) Parents(W)) given Parents(W). (Spirtes et al. 2000: 11) or, in other words: Any vertex (node) is conditionally independent of its nondescendants (except parents), given its parents. Causality in Econometrics 22/27
23 Markov Condition (example) V 1 V2 V4 V5 V 3 The DAG above and the probability distribution P(V 1, V 2, V 3, V 4 ) satisfy MC iff: (1) V 4 {V 1, V 3 } V 2 (2) V 5 {V 1, V 2, V 3 } V 4 Notice that many other c.i. relations follow from (1) and (2) by applying symmetry, decomposition, and weak union (see Slide 13 ). For example {V 1, V 3 } V 4 V 2 ; V 1 V 4 V 2 ; V 3 V 4 V 2 ; V 1, V 4 {V 2, V 3 }; etc. {V 1, V 2, V 3 } V 5 V 4 ; V 5 {V 1, V 2 } V 4 ; etc. Causality in Econometrics 23/27
24 Markov condition (factorization) The M.C. permits the following factorization: discrete case: P(V 1,..., V n ) = Π n i=1 P(V i Parents(V i )),where if Parents(V i ) =, P(V i Parents(V i )) = P(V i ) continuous case: f (V 1,..., V n ) = Π n i=1 f (V i Parents(V i )), where if Parents(V i ) =, f (V i Parents(V i )) = f (V i ) V 1 V2 V4 V5 V 3 We have: P(V 1, V 2, V 3, V 4, V 5 ) = P(V 1 V 3 )P(V 2 V 1, V 3 )P(V 3 )P(V 4 V 2 )P(V 5 V 4 ) Recall chain rule: in general P(V 1,..., V n ) = P(V n V n 1,..., V 2, V 1 ),..., P(V 2 V 1 )P(V 1 ) Causality in Econometrics 24/27
25 The d-separation criterion d-separation: a graphical criterion which captures exactly all the C.I. relationships that are implied by the M.C. Consider a graph G, with distinct nodes X, Y and a set of nodes W, where neither X nor Y belongs to W. We say that X and Y are d-separated given W in G iff there exists no undirected path U between X and Y, such that: 1 every collider C ( C ) on U is in W or has a descendant in W, and 2 no other vertex on U is in W. if there is such a path, then X and Y are d-connected. (cfr. Spirtes et al. 2000: 14). Included those derived by the MC through symmetry, decomposition and weak union. Causality in Econometrics 25/27
26 The d-separation criterion (Pearl s definition) d-separation: Consider a graph G, with distinct nodes X, Y and a set of nodes W, where neither X nor Y belongs to W. A path U is said to be d-separated by a set of nodes W iff 1 U contains a chain ( C or C ) or a fork ( C ) such that the middle node C W, or 2 U contains a collider C ( C ) s.t. C / W and s.t. no descendant of C is in W. A set W is said to d-separate X from Y iff W every path from X to Y is d-separated by W. Otherwise X and Y d-connected by W. (cfr. Pearl 2000: 16-17). Causality in Econometrics 26/27
27 Graphical Causal Models References Reading List Spirtes, Glymour, Scheines (2000), Causation, Prediction, and Search, MIT Press 2 nd edition: Chapter 1 and 2 Pearl (2000), Causality: Models, Reasoning, and Inference, CUP: Section 1.1 and 1.2 Spanos, A. (1999), Probability Theory and Statistical Inference. CUP: Further reading: Section 2.2 and 6.4 Cooper, G.F. (1999), An Overview of the Representation and Discovery of Causal Relationships Using Bayesian Networks, in C. Glymour, G.F. Cooper, Computation Causation, and Discovery, MIT Press. Scheines, R. (1997), An Introduction to causal inference. www Causality in Econometrics 27/27
Graphical models and causality: Directed acyclic graphs (DAGs) and conditional (in)dependence
Graphical models and causality: Directed acyclic graphs (DAGs) and conditional (in)dependence General overview Introduction Directed acyclic graphs (DAGs) and conditional independence DAGs and causal effects
More information1. what conditional independencies are implied by the graph. 2. whether these independecies correspond to the probability distribution
NETWORK ANALYSIS Lourens Waldorp PROBABILITY AND GRAPHS The objective is to obtain a correspondence between the intuitive pictures (graphs) of variables of interest and the probability distributions of
More informationEquivalence in Non-Recursive Structural Equation Models
Equivalence in Non-Recursive Structural Equation Models Thomas Richardson 1 Philosophy Department, Carnegie-Mellon University Pittsburgh, P 15213, US thomas.richardson@andrew.cmu.edu Introduction In the
More informationCausal Inference & Reasoning with Causal Bayesian Networks
Causal Inference & Reasoning with Causal Bayesian Networks Neyman-Rubin Framework Potential Outcome Framework: for each unit k and each treatment i, there is a potential outcome on an attribute U, U ik,
More informationAutomatic Causal Discovery
Automatic Causal Discovery Richard Scheines Peter Spirtes, Clark Glymour Dept. of Philosophy & CALD Carnegie Mellon 1 Outline 1. Motivation 2. Representation 3. Discovery 4. Using Regression for Causal
More informationLecture 4 October 18th
Directed and undirected graphical models Fall 2017 Lecture 4 October 18th Lecturer: Guillaume Obozinski Scribe: In this lecture, we will assume that all random variables are discrete, to keep notations
More informationSummary of the Bayes Net Formalism. David Danks Institute for Human & Machine Cognition
Summary of the Bayes Net Formalism David Danks Institute for Human & Machine Cognition Bayesian Networks Two components: 1. Directed Acyclic Graph (DAG) G: There is a node for every variable D: Some nodes
More informationIdentifiability assumptions for directed graphical models with feedback
Biometrika, pp. 1 26 C 212 Biometrika Trust Printed in Great Britain Identifiability assumptions for directed graphical models with feedback BY GUNWOONG PARK Department of Statistics, University of Wisconsin-Madison,
More informationRapid Introduction to Machine Learning/ Deep Learning
Rapid Introduction to Machine Learning/ Deep Learning Hyeong In Choi Seoul National University 1/32 Lecture 5a Bayesian network April 14, 2016 2/32 Table of contents 1 1. Objectives of Lecture 5a 2 2.Bayesian
More informationChapter 16. Structured Probabilistic Models for Deep Learning
Peng et al.: Deep Learning and Practice 1 Chapter 16 Structured Probabilistic Models for Deep Learning Peng et al.: Deep Learning and Practice 2 Structured Probabilistic Models way of using graphs to describe
More informationUsing Descendants as Instrumental Variables for the Identification of Direct Causal Effects in Linear SEMs
Using Descendants as Instrumental Variables for the Identification of Direct Causal Effects in Linear SEMs Hei Chan Institute of Statistical Mathematics, 10-3 Midori-cho, Tachikawa, Tokyo 190-8562, Japan
More informationCausal Reasoning with Ancestral Graphs
Journal of Machine Learning Research 9 (2008) 1437-1474 Submitted 6/07; Revised 2/08; Published 7/08 Causal Reasoning with Ancestral Graphs Jiji Zhang Division of the Humanities and Social Sciences California
More informationTowards an extension of the PC algorithm to local context-specific independencies detection
Towards an extension of the PC algorithm to local context-specific independencies detection Feb-09-2016 Outline Background: Bayesian Networks The PC algorithm Context-specific independence: from DAGs to
More informationArrowhead completeness from minimal conditional independencies
Arrowhead completeness from minimal conditional independencies Tom Claassen, Tom Heskes Radboud University Nijmegen The Netherlands {tomc,tomh}@cs.ru.nl Abstract We present two inference rules, based on
More informationChris Bishop s PRML Ch. 8: Graphical Models
Chris Bishop s PRML Ch. 8: Graphical Models January 24, 2008 Introduction Visualize the structure of a probabilistic model Design and motivate new models Insights into the model s properties, in particular
More informationAbstract. Three Methods and Their Limitations. N-1 Experiments Suffice to Determine the Causal Relations Among N Variables
N-1 Experiments Suffice to Determine the Causal Relations Among N Variables Frederick Eberhardt Clark Glymour 1 Richard Scheines Carnegie Mellon University Abstract By combining experimental interventions
More informationIntroduction to Causal Calculus
Introduction to Causal Calculus Sanna Tyrväinen University of British Columbia August 1, 2017 1 / 1 2 / 1 Bayesian network Bayesian networks are Directed Acyclic Graphs (DAGs) whose nodes represent random
More informationCS Lecture 3. More Bayesian Networks
CS 6347 Lecture 3 More Bayesian Networks Recap Last time: Complexity challenges Representing distributions Computing probabilities/doing inference Introduction to Bayesian networks Today: D-separation,
More informationIntroduction to Probabilistic Graphical Models
Introduction to Probabilistic Graphical Models Kyu-Baek Hwang and Byoung-Tak Zhang Biointelligence Lab School of Computer Science and Engineering Seoul National University Seoul 151-742 Korea E-mail: kbhwang@bi.snu.ac.kr
More informationCausal Discovery. Richard Scheines. Peter Spirtes, Clark Glymour, and many others. Dept. of Philosophy & CALD Carnegie Mellon
Causal Discovery Richard Scheines Peter Spirtes, Clark Glymour, and many others Dept. of Philosophy & CALD Carnegie Mellon Graphical Models --11/30/05 1 Outline 1. Motivation 2. Representation 3. Connecting
More informationCS 2750: Machine Learning. Bayesian Networks. Prof. Adriana Kovashka University of Pittsburgh March 14, 2016
CS 2750: Machine Learning Bayesian Networks Prof. Adriana Kovashka University of Pittsburgh March 14, 2016 Plan for today and next week Today and next time: Bayesian networks (Bishop Sec. 8.1) Conditional
More informationLearning causal network structure from multiple (in)dependence models
Learning causal network structure from multiple (in)dependence models Tom Claassen Radboud University, Nijmegen tomc@cs.ru.nl Abstract Tom Heskes Radboud University, Nijmegen tomh@cs.ru.nl We tackle the
More informationTópicos Especiais em Modelagem e Análise - Aprendizado por Máquina CPS863
Tópicos Especiais em Modelagem e Análise - Aprendizado por Máquina CPS863 Daniel, Edmundo, Rosa Terceiro trimestre de 2012 UFRJ - COPPE Programa de Engenharia de Sistemas e Computação Bayesian Networks
More informationDirected Graphical Models
CS 2750: Machine Learning Directed Graphical Models Prof. Adriana Kovashka University of Pittsburgh March 28, 2017 Graphical Models If no assumption of independence is made, must estimate an exponential
More informationDetecting marginal and conditional independencies between events and learning their causal structure.
Detecting marginal and conditional independencies between events and learning their causal structure. Jan Lemeire 1,4, Stijn Meganck 1,3, Albrecht Zimmermann 2, and Thomas Dhollander 3 1 ETRO Department,
More informationProbabilistic Graphical Networks: Definitions and Basic Results
This document gives a cursory overview of Probabilistic Graphical Networks. The material has been gleaned from different sources. I make no claim to original authorship of this material. Bayesian Graphical
More informationLearning in Bayesian Networks
Learning in Bayesian Networks Florian Markowetz Max-Planck-Institute for Molecular Genetics Computational Molecular Biology Berlin Berlin: 20.06.2002 1 Overview 1. Bayesian Networks Stochastic Networks
More informationSupplementary material to Structure Learning of Linear Gaussian Structural Equation Models with Weak Edges
Supplementary material to Structure Learning of Linear Gaussian Structural Equation Models with Weak Edges 1 PRELIMINARIES Two vertices X i and X j are adjacent if there is an edge between them. A path
More informationCovariance and Correlation
Covariance and Correlation ST 370 The probability distribution of a random variable gives complete information about its behavior, but its mean and variance are useful summaries. Similarly, the joint probability
More informationDirected and Undirected Graphical Models
Directed and Undirected Davide Bacciu Dipartimento di Informatica Università di Pisa bacciu@di.unipi.it Machine Learning: Neural Networks and Advanced Models (AA2) Last Lecture Refresher Lecture Plan Directed
More informationGeneralized Measurement Models
Generalized Measurement Models Ricardo Silva and Richard Scheines Center for Automated Learning and Discovery rbas@cs.cmu.edu, scheines@andrew.cmu.edu May 16, 2005 CMU-CALD-04-101 School of Computer Science
More informationCausal Effect Identification in Alternative Acyclic Directed Mixed Graphs
Proceedings of Machine Learning Research vol 73:21-32, 2017 AMBN 2017 Causal Effect Identification in Alternative Acyclic Directed Mixed Graphs Jose M. Peña Linköping University Linköping (Sweden) jose.m.pena@liu.se
More informationLearning Multivariate Regression Chain Graphs under Faithfulness
Sixth European Workshop on Probabilistic Graphical Models, Granada, Spain, 2012 Learning Multivariate Regression Chain Graphs under Faithfulness Dag Sonntag ADIT, IDA, Linköping University, Sweden dag.sonntag@liu.se
More information2 : Directed GMs: Bayesian Networks
10-708: Probabilistic Graphical Models 10-708, Spring 2017 2 : Directed GMs: Bayesian Networks Lecturer: Eric P. Xing Scribes: Jayanth Koushik, Hiroaki Hayashi, Christian Perez Topic: Directed GMs 1 Types
More informationProbabilistic Graphical Models (I)
Probabilistic Graphical Models (I) Hongxin Zhang zhx@cad.zju.edu.cn State Key Lab of CAD&CG, ZJU 2015-03-31 Probabilistic Graphical Models Modeling many real-world problems => a large number of random
More informationConditional Independence and Factorization
Conditional Independence and Factorization Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr
More informationOn the Identification of a Class of Linear Models
On the Identification of a Class of Linear Models Jin Tian Department of Computer Science Iowa State University Ames, IA 50011 jtian@cs.iastate.edu Abstract This paper deals with the problem of identifying
More informationFaithfulness of Probability Distributions and Graphs
Journal of Machine Learning Research 18 (2017) 1-29 Submitted 5/17; Revised 11/17; Published 12/17 Faithfulness of Probability Distributions and Graphs Kayvan Sadeghi Statistical Laboratory University
More informationCausal Bayesian networks. Peter Antal
Causal Bayesian networks Peter Antal antal@mit.bme.hu A.I. 4/8/2015 1 Can we represent exactly (in)dependencies by a BN? From a causal model? Suff.&nec.? Can we interpret edges as causal relations with
More information6.047 / Computational Biology: Genomes, Networks, Evolution Fall 2008
MIT OpenCourseWare http://ocw.mit.edu 6.047 / 6.878 Computational Biology: Genomes, Networks, Evolution Fall 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
More informationCSC 412 (Lecture 4): Undirected Graphical Models
CSC 412 (Lecture 4): Undirected Graphical Models Raquel Urtasun University of Toronto Feb 2, 2016 R Urtasun (UofT) CSC 412 Feb 2, 2016 1 / 37 Today Undirected Graphical Models: Semantics of the graph:
More informationRecall from last time. Lecture 3: Conditional independence and graph structure. Example: A Bayesian (belief) network.
ecall from last time Lecture 3: onditional independence and graph structure onditional independencies implied by a belief network Independence maps (I-maps) Factorization theorem The Bayes ball algorithm
More informationDirected Graphical Models or Bayesian Networks
Directed Graphical Models or Bayesian Networks Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012 Bayesian Networks One of the most exciting recent advancements in statistical AI Compact
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear
More informationMarkov properties for undirected graphs
Graphical Models, Lecture 2, Michaelmas Term 2011 October 12, 2011 Formal definition Fundamental properties Random variables X and Y are conditionally independent given the random variable Z if L(X Y,
More informationLearning Semi-Markovian Causal Models using Experiments
Learning Semi-Markovian Causal Models using Experiments Stijn Meganck 1, Sam Maes 2, Philippe Leray 2 and Bernard Manderick 1 1 CoMo Vrije Universiteit Brussel Brussels, Belgium 2 LITIS INSA Rouen St.
More informationCausal Bayesian networks. Peter Antal
Causal Bayesian networks Peter Antal antal@mit.bme.hu A.I. 11/25/2015 1 Can we represent exactly (in)dependencies by a BN? From a causal model? Suff.&nec.? Can we interpret edges as causal relations with
More informationSTATISTICAL METHODS IN AI/ML Vibhav Gogate The University of Texas at Dallas. Bayesian networks: Representation
STATISTICAL METHODS IN AI/ML Vibhav Gogate The University of Texas at Dallas Bayesian networks: Representation Motivation Explicit representation of the joint distribution is unmanageable Computationally:
More information4.1 Notation and probability review
Directed and undirected graphical models Fall 2015 Lecture 4 October 21st Lecturer: Simon Lacoste-Julien Scribe: Jaime Roquero, JieYing Wu 4.1 Notation and probability review 4.1.1 Notations Let us recall
More informationThe Role of Assumptions in Causal Discovery
The Role of Assumptions in Causal Discovery Marek J. Druzdzel Decision Systems Laboratory, School of Information Sciences and Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA 15260,
More informationLearning the Structure of Linear Latent Variable Models
Journal (2005) - Submitted /; Published / Learning the Structure of Linear Latent Variable Models Ricardo Silva Center for Automated Learning and Discovery School of Computer Science rbas@cs.cmu.edu Richard
More informationIntelligent Systems: Reasoning and Recognition. Reasoning with Bayesian Networks
Intelligent Systems: Reasoning and Recognition James L. Crowley ENSIMAG 2 / MoSIG M1 Second Semester 2016/2017 Lesson 13 24 march 2017 Reasoning with Bayesian Networks Naïve Bayesian Systems...2 Example
More informationMarkov Independence (Continued)
Markov Independence (Continued) As an Undirected Graph Basic idea: Each variable V is represented as a vertex in an undirected graph G = (V(G), E(G)), with set of vertices V(G) and set of edges E(G) the
More informationCheng Soon Ong & Christian Walder. Canberra February June 2018
Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 2018 Outlines Overview Introduction Linear Algebra Probability Linear Regression
More informationLearning Marginal AMP Chain Graphs under Faithfulness
Learning Marginal AMP Chain Graphs under Faithfulness Jose M. Peña ADIT, IDA, Linköping University, SE-58183 Linköping, Sweden jose.m.pena@liu.se Abstract. Marginal AMP chain graphs are a recently introduced
More informationLearning Causal Bayesian Networks from Observations and Experiments: A Decision Theoretic Approach
Learning Causal Bayesian Networks from Observations and Experiments: A Decision Theoretic Approach Stijn Meganck 1, Philippe Leray 2, and Bernard Manderick 1 1 Vrije Universiteit Brussel, Pleinlaan 2,
More informationReasoning with Bayesian Networks
Reasoning with Lecture 1: Probability Calculus, NICTA and ANU Reasoning with Overview of the Course Probability calculus, Bayesian networks Inference by variable elimination, factor elimination, conditioning
More informationIdentifying Linear Causal Effects
Identifying Linear Causal Effects Jin Tian Department of Computer Science Iowa State University Ames, IA 50011 jtian@cs.iastate.edu Abstract This paper concerns the assessment of linear cause-effect relationships
More informationA Brief Introduction to Graphical Models. Presenter: Yijuan Lu November 12,2004
A Brief Introduction to Graphical Models Presenter: Yijuan Lu November 12,2004 References Introduction to Graphical Models, Kevin Murphy, Technical Report, May 2001 Learning in Graphical Models, Michael
More informationCOMP538: Introduction to Bayesian Networks
COMP538: Introduction to Bayesian Networks Lecture 9: Optimal Structure Learning Nevin L. Zhang lzhang@cse.ust.hk Department of Computer Science and Engineering Hong Kong University of Science and Technology
More informationFinal Exam. Economics 835: Econometrics. Fall 2010
Final Exam Economics 835: Econometrics Fall 2010 Please answer the question I ask - no more and no less - and remember that the correct answer is often short and simple. 1 Some short questions a) For each
More informationExchangeability and Invariance: A Causal Theory. Jiji Zhang. (Very Preliminary Draft) 1. Motivation: Lindley-Novick s Puzzle
Exchangeability and Invariance: A Causal Theory Jiji Zhang (Very Preliminary Draft) 1. Motivation: Lindley-Novick s Puzzle In their seminal paper on the role of exchangeability in statistical inference,
More informationBayesian Networks Representation
Bayesian Networks Representation Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University March 19 th, 2007 Handwriting recognition Character recognition, e.g., kernel SVMs a c z rr r r
More information1 : Introduction. 1 Course Overview. 2 Notation. 3 Representing Multivariate Distributions : Probabilistic Graphical Models , Spring 2014
10-708: Probabilistic Graphical Models 10-708, Spring 2014 1 : Introduction Lecturer: Eric P. Xing Scribes: Daniel Silva and Calvin McCarter 1 Course Overview In this lecture we introduce the concept of
More informationA graph contains a set of nodes (vertices) connected by links (edges or arcs)
BOLTZMANN MACHINES Generative Models Graphical Models A graph contains a set of nodes (vertices) connected by links (edges or arcs) In a probabilistic graphical model, each node represents a random variable,
More informationStatistical Approaches to Learning and Discovery
Statistical Approaches to Learning and Discovery Graphical Models Zoubin Ghahramani & Teddy Seidenfeld zoubin@cs.cmu.edu & teddy@stat.cmu.edu CALD / CS / Statistics / Philosophy Carnegie Mellon University
More informationReasoning Under Uncertainty: Belief Network Inference
Reasoning Under Uncertainty: Belief Network Inference CPSC 322 Uncertainty 5 Textbook 10.4 Reasoning Under Uncertainty: Belief Network Inference CPSC 322 Uncertainty 5, Slide 1 Lecture Overview 1 Recap
More informationCausal Models with Hidden Variables
Causal Models with Hidden Variables Robin J. Evans www.stats.ox.ac.uk/ evans Department of Statistics, University of Oxford Quantum Networks, Oxford August 2017 1 / 44 Correlation does not imply causation
More informationCarnegie Mellon Pittsburgh, Pennsylvania 15213
A Tutorial On Causal Inference Peter Spirtes August 4, 2009 Technical Report No. CMU-PHIL-183 Philosophy Methodology Logic Carnegie Mellon Pittsburgh, Pennsylvania 15213 1. Introduction A Tutorial On Causal
More informationPart I. C. M. Bishop PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 8: GRAPHICAL MODELS
Part I C. M. Bishop PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 8: GRAPHICAL MODELS Probabilistic Graphical Models Graphical representation of a probabilistic model Each variable corresponds to a
More informationCS839: Probabilistic Graphical Models. Lecture 2: Directed Graphical Models. Theo Rekatsinas
CS839: Probabilistic Graphical Models Lecture 2: Directed Graphical Models Theo Rekatsinas 1 Questions Questions? Waiting list Questions on other logistics 2 Section 1 1. Intro to Bayes Nets 3 Section
More informationBayesian networks as causal models. Peter Antal
Bayesian networks as causal models Peter Antal antal@mit.bme.hu A.I. 3/20/2018 1 Can we represent exactly (in)dependencies by a BN? From a causal model? Suff.&nec.? Can we interpret edges as causal relations
More informationBelief Update in CLG Bayesian Networks With Lazy Propagation
Belief Update in CLG Bayesian Networks With Lazy Propagation Anders L Madsen HUGIN Expert A/S Gasværksvej 5 9000 Aalborg, Denmark Anders.L.Madsen@hugin.com Abstract In recent years Bayesian networks (BNs)
More informationThe Two Sides of Interventionist Causation
The Two Sides of Interventionist Causation Peter W. Evans and Sally Shrapnel School of Historical and Philosophical Inquiry University of Queensland Australia Abstract Pearl and Woodward are both well-known
More informationCausal Search in Time Series Models
Graphs SEM SEM-VAR SVAR Empirical application Gains, pitfalls & challenges References Causal Search in Time Series Models Alessio Moneta Institute of Economics Scuola Superiore Sant Anna, Pisa amoneta@sssup.it
More informationUndirected Graphical Models
Undirected Graphical Models 1 Conditional Independence Graphs Let G = (V, E) be an undirected graph with vertex set V and edge set E, and let A, B, and C be subsets of vertices. We say that C separates
More informationInterpreting and using CPDAGs with background knowledge
Interpreting and using CPDAGs with background knowledge Emilija Perković Seminar for Statistics ETH Zurich, Switzerland perkovic@stat.math.ethz.ch Markus Kalisch Seminar for Statistics ETH Zurich, Switzerland
More informationPreliminaries Bayesian Networks Graphoid Axioms d-separation Wrap-up. Bayesian Networks. Brandon Malone
Preliminaries Graphoid Axioms d-separation Wrap-up Much of this material is adapted from Chapter 4 of Darwiche s book January 23, 2014 Preliminaries Graphoid Axioms d-separation Wrap-up 1 Preliminaries
More informationECE521 Tutorial 11. Topic Review. ECE521 Winter Credits to Alireza Makhzani, Alex Schwing, Rich Zemel and TAs for slides. ECE521 Tutorial 11 / 4
ECE52 Tutorial Topic Review ECE52 Winter 206 Credits to Alireza Makhzani, Alex Schwing, Rich Zemel and TAs for slides ECE52 Tutorial ECE52 Winter 206 Credits to Alireza / 4 Outline K-means, PCA 2 Bayesian
More informationANALYTIC COMPARISON. Pearl and Rubin CAUSAL FRAMEWORKS
ANALYTIC COMPARISON of Pearl and Rubin CAUSAL FRAMEWORKS Content Page Part I. General Considerations Chapter 1. What is the question? 16 Introduction 16 1. Randomization 17 1.1 An Example of Randomization
More informationLecture 1: Bayesian Framework Basics
Lecture 1: Bayesian Framework Basics Melih Kandemir melih.kandemir@iwr.uni-heidelberg.de April 21, 2014 What is this course about? Building Bayesian machine learning models Performing the inference of
More informationTópicos Especiais em Modelagem e Análise - Aprendizado por Máquina CPS863
Tópicos Especiais em Modelagem e Análise - Aprendizado por Máquina CPS863 Daniel, Edmundo, Rosa Terceiro trimestre de 2012 UFRJ - COPPE Programa de Engenharia de Sistemas e Computação Motivação Grandes
More informationMachine Learning Lecture 14
Many slides adapted from B. Schiele, S. Roth, Z. Gharahmani Machine Learning Lecture 14 Undirected Graphical Models & Inference 23.06.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de/ leibe@vision.rwth-aachen.de
More information3 : Representation of Undirected GM
10-708: Probabilistic Graphical Models 10-708, Spring 2016 3 : Representation of Undirected GM Lecturer: Eric P. Xing Scribes: Longqi Cai, Man-Chia Chang 1 MRF vs BN There are two types of graphical models:
More informationTutorial: Causal Model Search
Tutorial: Causal Model Search Richard Scheines Carnegie Mellon University Peter Spirtes, Clark Glymour, Joe Ramsey, others 1 Goals 1) Convey rudiments of graphical causal models 2) Basic working knowledge
More informationMotivation. Bayesian Networks in Epistemology and Philosophy of Science Lecture. Overview. Organizational Issues
Bayesian Networks in Epistemology and Philosophy of Science Lecture 1: Bayesian Networks Center for Logic and Philosophy of Science Tilburg University, The Netherlands Formal Epistemology Course Northern
More informationLearning from Sensor Data: Set II. Behnaam Aazhang J.S. Abercombie Professor Electrical and Computer Engineering Rice University
Learning from Sensor Data: Set II Behnaam Aazhang J.S. Abercombie Professor Electrical and Computer Engineering Rice University 1 6. Data Representation The approach for learning from data Probabilistic
More informationCausal Discovery and MIMIC Models. Alexander Murray-Watters
Causal Discovery and MIMIC Models Alexander Murray-Watters April 30, 2013 Contents List of Figures iii 1 Introduction 1 2 Motivation 4 2.1 Place in the Method of Science................... 4 2.2 MIMIC
More information2 : Directed GMs: Bayesian Networks
10-708: Probabilistic Graphical Models, Spring 2015 2 : Directed GMs: Bayesian Networks Lecturer: Eric P. Xing Scribes: Yi Cheng, Cong Lu 1 Notation Here the notations used in this course are defined:
More informationEconometrics A. Simple linear model (2) Keio University, Faculty of Economics. Simon Clinet (Keio University) Econometrics A October 16, / 11
Econometrics A Keio University, Faculty of Economics Simple linear model (2) Simon Clinet (Keio University) Econometrics A October 16, 2018 1 / 11 Estimation of the noise variance σ 2 In practice σ 2 too
More informationDay 3: Search Continued
Center for Causal Discovery Day 3: Search Continued June 15, 2015 Carnegie Mellon University 1 Outline Models Data 1) Bridge Principles: Markov Axiom and D-separation 2) Model Equivalence 3) Model Search
More informationMA 575 Linear Models: Cedric E. Ginestet, Boston University Revision: Probability and Linear Algebra Week 1, Lecture 2
MA 575 Linear Models: Cedric E Ginestet, Boston University Revision: Probability and Linear Algebra Week 1, Lecture 2 1 Revision: Probability Theory 11 Random Variables A real-valued random variable is
More informationSimple Linear Regression: The Model
Simple Linear Regression: The Model task: quantifying the effect of change X in X on Y, with some constant β 1 : Y = β 1 X, linear relationship between X and Y, however, relationship subject to a random
More informationBayesian Graphical Models for Structural Vector AutoregressiveMarch Processes 21, / 1
Bayesian Graphical Models for Structural Vector Autoregressive Processes Daniel Ahelegbey, Monica Billio, and Roberto Cassin (2014) March 21, 2015 Bayesian Graphical Models for Structural Vector AutoregressiveMarch
More informationProbabilistic Graphical Models. Rudolf Kruse, Alexander Dockhorn Bayesian Networks 153
Probabilistic Graphical Models Rudolf Kruse, Alexander Dockhorn Bayesian Networks 153 The Big Objective(s) In a wide variety of application fields two main problems need to be addressed over and over:
More informationLocal Characterizations of Causal Bayesian Networks
In M. Croitoru, S. Rudolph, N. Wilson, J. Howse, and O. Corby (Eds.), GKR 2011, LNAI 7205, Berlin Heidelberg: Springer-Verlag, pp. 1-17, 2012. TECHNICAL REPORT R-384 May 2011 Local Characterizations of
More informationBased on slides by Richard Zemel
CSC 412/2506 Winter 2018 Probabilistic Learning and Reasoning Lecture 3: Directed Graphical Models and Latent Variables Based on slides by Richard Zemel Learning outcomes What aspects of a model can we
More informationTutorial: Gaussian conditional independence and graphical models. Thomas Kahle Otto-von-Guericke Universität Magdeburg
Tutorial: Gaussian conditional independence and graphical models Thomas Kahle Otto-von-Guericke Universität Magdeburg The central dogma of algebraic statistics Statistical models are varieties The central
More informationChapter 17: Undirected Graphical Models
Chapter 17: Undirected Graphical Models The Elements of Statistical Learning Biaobin Jiang Department of Biological Sciences Purdue University bjiang@purdue.edu October 30, 2014 Biaobin Jiang (Purdue)
More informationCAUSAL MODELS: THE MEANINGFUL INFORMATION OF PROBABILITY DISTRIBUTIONS
CAUSAL MODELS: THE MEANINGFUL INFORMATION OF PROBABILITY DISTRIBUTIONS Jan Lemeire, Erik Dirkx ETRO Dept., Vrije Universiteit Brussel Pleinlaan 2, 1050 Brussels, Belgium jan.lemeire@vub.ac.be ABSTRACT
More information