Learning from Sensor Data: Set II. Behnaam Aazhang J.S. Abercombie Professor Electrical and Computer Engineering Rice University

Similar documents
Chris Bishop s PRML Ch. 8: Graphical Models

Chapter 16. Structured Probabilistic Models for Deep Learning

Lecture 4 October 18th

Part I. C. M. Bishop PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 8: GRAPHICAL MODELS

4.1 Notation and probability review

CS 2750: Machine Learning. Bayesian Networks. Prof. Adriana Kovashka University of Pittsburgh March 14, 2016

Graphical Models and Kernel Methods

{ p if x = 1 1 p if x = 0

CPSC 540: Machine Learning

Graphical models and causality: Directed acyclic graphs (DAGs) and conditional (in)dependence

MATH 829: Introduction to Data Mining and Analysis Graphical Models I

Machine Learning 4771

CSC 412 (Lecture 4): Undirected Graphical Models

Probabilistic Graphical Models

Undirected Graphical Models

Directed Graphical Models

Directed and Undirected Graphical Models

1 T 1 = where 1 is the all-ones vector. For the upper bound, let v 1 be the eigenvector corresponding. u:(u,v) E v 1(u)

Independencies. Undirected Graphical Models 2: Independencies. Independencies (Markov networks) Independencies (Bayesian Networks)

Message Passing and Junction Tree Algorithms. Kayhan Batmanghelich

Directed Graphical Models or Bayesian Networks

Machine Learning Lecture Notes

MATH 829: Introduction to Data Mining and Analysis Clustering II

Probabilistic Graphical Models (I)

STATISTICAL METHODS IN AI/ML Vibhav Gogate The University of Texas at Dallas. Bayesian networks: Representation

Conditional Independence

Based on slides by Richard Zemel

Prof. Dr. Lars Schmidt-Thieme, L. B. Marinho, K. Buza Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany, Course

Recall from last time. Lecture 3: Conditional independence and graph structure. Example: A Bayesian (belief) network.

Cheng Soon Ong & Christian Walder. Canberra February June 2018

A lower bound for the Laplacian eigenvalues of a graph proof of a conjecture by Guo

Spectra of Adjacency and Laplacian Matrices

CS Lecture 4. Markov Random Fields

Basic graph theory 18.S995 - L26.

Data Mining 2018 Bayesian Networks (1)

Graph fundamentals. Matrices associated with a graph

Multivariate random variables

3 : Representation of Undirected GM

Graphical Models - Part I

Recitation 2: Probability

Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science Algorithms For Inference Fall 2014

Review of Probability Theory

Probabilistic Graphical Models

Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science Algorithms For Inference Fall 2014

Markov Independence (Continued)

1. what conditional independencies are implied by the graph. 2. whether these independecies correspond to the probability distribution

Intelligent Systems:

Undirected Graphical Models

ECE521 Tutorial 11. Topic Review. ECE521 Winter Credits to Alireza Makhzani, Alex Schwing, Rich Zemel and TAs for slides. ECE521 Tutorial 11 / 4

Spectral Graph Theory and You: Matrix Tree Theorem and Centrality Metrics

Graphical Models and Independence Models

STA 4273H: Statistical Machine Learning

Sistemi Cognitivi per le

Implementing Machine Reasoning using Bayesian Network in Big Data Analytics

The Signless Laplacian Spectral Radius of Graphs with Given Degree Sequences. Dedicated to professor Tian Feng on the occasion of his 70 birthday

Spectral Clustering. Guokun Lai 2016/10

Preliminaries and Complexity Theory

A New Spectral Technique Using Normalized Adjacency Matrices for Graph Matching 1

Algebraic Representation of Networks

Multivariate random variables

Bayes Nets: Independence

Markov Chains and Spectral Clustering

Alternative Parameterizations of Markov Networks. Sargur Srihari

Exact Inference I. Mark Peot. In this lecture we will look at issues associated with exact inference. = =

Basics on Probability. Jingrui He 09/11/2007

Undirected Graphical Models: Markov Random Fields

1 Undirected Graphical Models. 2 Markov Random Fields (MRFs)

A graph contains a set of nodes (vertices) connected by links (edges or arcs)

Representation of undirected GM. Kayhan Batmanghelich

8.1 Concentration inequality for Gaussian random matrix (cont d)

Machine Learning Summer School

Probability & Computing

Linear algebra and applications to graphs Part 1

Undirected graphical models

Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5

Lecture 1: Basics of Probability

Lecture 15. Probabilistic Models on Graph

6.047 / Computational Biology: Genomes, Networks, Evolution Fall 2008

Markov Chains and MCMC

CSE 473: Artificial Intelligence Autumn 2011

Part I Qualitative Probabilistic Networks

Lecture 19: NP-Completeness 1

Lecture 9: PGM Learning

Fiedler s Theorems on Nodal Domains

Causality in Econometrics (3)

CS 5522: Artificial Intelligence II

Machine Learning Lecture 14

Alternative Parameterizations of Markov Networks. Sargur Srihari

6.867 Machine learning, lecture 23 (Jaakkola)

CS6220: DATA MINING TECHNIQUES

Product rule. Chain rule

Statistical Approaches to Learning and Discovery

Codes on graphs. Chapter Elementary realizations of linear block codes

p L yi z n m x N n xi

Markov properties for undirected graphs

CS281A/Stat241A Lecture 19

9 Forward-backward algorithm, sum-product on factor graphs

ACO Comprehensive Exam March 17 and 18, Computability, Complexity and Algorithms

Graphical Model Inference with Perfect Graphs

Introduction to Graphical Models

Transcription:

Learning from Sensor Data: Set II Behnaam Aazhang J.S. Abercombie Professor Electrical and Computer Engineering Rice University 1

6. Data Representation The approach for learning from data Probabilistic modeling and algebraic manipulation Diagrammatic representation is often extremely useful Probabilistic graphical modeling Visualize the structure Infer dependence based on inspection of the graph Simplify complex computations 2

Examples of graphical modeling in engineering problems Circuit diagrams Signal flow diagrams Trellis diagrams Block diagrams 3

A graph can be viewed as the simplest way to represent a complex system where vertices are simplest units of the system edges represent their mutual interactions 4

Elements Nodes or vertices X 4 X 5 A random variable (data) or a group of random variables X 6 X 7 Links or edges Probabilistic relationships between the variables 5

Examples of graphs C X1, I( ; ) I(! ) 6

A typical graph representing data RAH1 RPBT1 LAH12 RPH4 RMOF10 RAH13 LAH8 RPBT11 RMOF5 RAMY7 LAH5 RAH5 RAINS4 RAMY6 RAH2 RAMY12 RPH3 RAMY5 RAH3 RPH2 RAMY11 RPH10 RAH10 RAH6 RAMY10 RAMY9 RAMY2 RAINS3 RAMY4 RAMY3 7

RAH2 node is influencing several nodes RAH1 RPBT1 RPH4 RMOF10 LAH12 RAH13 LAH8 RPBT11 RMOF5 RAMY7 LAH5 RAH5 RAINS4 RAMY6 RAH2 RAMY12 RPH3 RAMY5 RAH3 RPH2 RAMY11 RPH10 RAH10 RAH6 RAMY10 RAMY9 RAMY2 RAINS3 RAMY4 RAMY3 8

Another example 9

10

neurons spike

does neuron 3 excite neuron 8?

does neuron 3 excite neuron 8?

did neuron 3 causally influence firing of neuron 8? Neuron 3 Neuron 8 00011110000001001100001100000001100001 00000111100000010111100001100000001111

I(! X 8 ) 15

Learning from graphs Identifying important features of data from graphs A graph G = (V,E) with V as the set of vertices and E as the set of edges A graph is simple if it has no parallel edges and no loops Adjacent edges and adjacent vertices are defined as the terms suggest The degree of vertex v is d(v) as the number of edges with v as the end A pendant vertex is a vertex with degree 1. 16

A graph is called regular if all vertices have the same degree In an undirected graph each edge is an unordered pair of vertices ( u, v ) In a directed graph each edge is an ordered pair of vertices ( u, v ) 17

In degree of vertex v in a directed graph is the number of edges with v as the end Out degree of vertex v in a directed graph is the number of edges with as the tail An isolated vertex is one with degree 0. In degree and out degree 0 in a directed graph. 18

For undirected graph we define the following concepts and properties Some definitions can be extended to directed graphs Minimum degree of a graph Maximum degree of a graph (G) (G) 19

It can be shown that for a graph G = ( V, E ) with n vertices and m edges then nx d(v i )=2m i=1 A graph G = ( V, E ) is a subgraph of graph H = ( W, F ) if V is a subset of W and every edge in E is also an edge in F. A complete graph is a simple graph with all the possible edges A complete subgraph of graph G is called a clique. 20

The density of a graph G = ( V, E ) is defined as where n 2 = (G) = m n 2 n! 2!(n 2)! for n 2 The density of a complete graph is 1 The adjacency matrix of graph G is a n x n matrix 21

The spectrum of graph G = ( V, E ) is the set of eigenvalues of the adjacency matrix and their eigenvectors. The Laplacian matrix of graph G = ( V, E ) is defined as where the diagonal degree matrix, D is defined as 22

The normalized Laplacian is The Laplacian matrix carries some of the key properties of a graph. Since the adjacency and Laplacian matrices are symmetric their eigenvalues are real. The eigenvalues of the normalized Laplacian are in [0, 2]. This fact makes it convenient to compare the spectral properties of two graphs. 23

Two graphs are isomorphic if any two vertices of one are adjacent if and only if the equivalent vertices in the other graph are also adjacent f(a) = 1 f(b) = 6 f(c) = 8 f(d) = 3 f(g) = 5 f(h) = 2 f(i) = 4 f(j) = 7 24

Graphs that have the same spectrum are referred to as cospectral ( or isospectral) If two graphs have the same eigenvalues but different eigenvectors they are referred to as weakly cospectral. Although adjacency matrix of a graph depends on the labeling of the vertices, the spectrum of a graph is independent of labeling. Isomorphic graphs are cospectral but not all cospectral graphs are isomorphic 25

The complement of graph G = ( V, E ) is Ḡ =(V,Ē) where the edges in complement graph are the ones not in E Common binary and linear operations can be defined for graphs Complement, union, intersection, ring sum, examples of commutative and associative operations. 26

A community is a group of vertices that belong together according to some criterion that could be measured An example, a group of vertices where the density of edges between the vertices in the group is higher than the average edge density in the graph In some literature a community is also referred to as a module or a cluster. 27

6 4 7 5 Examples 5 4 1 3 4 1 3 3 4 1 2 3 4 5 2 2 4 5 6 1 2 2 3 6 7 1 2 3 Figure 2.4 Bipartite Graph 28

<latexit sha1_base64="xys+jdtlh65oyskhfer6en2tb7a=">aaacbnicbvdlssnafj3uv62vqetbbotqqutsbhvxdooygrgfnptjdnioth7o3agldofgx3hjqswt3+dov3gabthww71woodezu7xysevwnaxuzibx1hcki6xvlbx1jfmza0bfswsmodgipitjygmemgc4cbyk5ambj5gte/2yuw375lupaqvyrgznyd9kpucetbs19yt4i66k5dwrofyyuror8ehxbnsva0mejbyosmjhi2u+dnprtqjwahuekxathwdmxijnao2knusxwjcb0mfttunsccum2z3jpc+vnryj6tuehcm/t5isadumpd0zebgoka9sfif107ap3vthsyjsjbohvitgshc41bwj0tgqqw1ivry/vdmb0qscjq6kg7bnj55lji16lnvujou18/znipob+2hcrlrcaqjs9radqloat2hf/rqpbrpxpvxphktgpnonvod4+mbbtiwba==</latexit> <latexit sha1_base64="x7ulc+u1qtnmk88yvbshzn/ptrq=">aaab83icbvbns8naej3ur1q/qh69lbahgpskcuqt6mvjbwmlbsib7azdutne3u2hhp4olx5uvppnvplv3ky5aotjbh7vzbczz485u9q2v63cyura+kzxs7s1vbo7v94/efrrigl1scqj2faxopwj6mqmow3hkulq57tlj25nfmtmpwkrenctmhohhggwmik1kbwqoj9dtlb2aa9cswt2brrmnjxuieezv/7q9ioshfrowrfshceotzdiqrnhdfrqjorgmizwghymftikykuzo6foxch9fetstnaou39vpdhuahl6zjleeqgwvzn4n9djdhdlpuzeiaaczb8keo50hgyjod6tlgg+mqqtycytiayxxesbneombgfxy8verdeua/b9ravxk6drhcm4hio4caknuimmuedgcz7hfd6ssfvivvsf89gcle8cwh9ynz+ugi7p</latexit> 3 4 1 2 2 3 6 7 Example 6.1. D = 2 3 1 0 0 0 60 2 0 0 7 40 0 1 05 A = 0 0 0 0 2 3 0 1 0 0 61 0 1 0 7 40 1 0 05 L = 0 0 0 0 2 6 4 3 1 1 0 0 1 2 1 0 7 0 1 1 05 0 0 0 0 The eigenvalues of adjacency matrix ( p 2, 0, 0, p 2) The eigenvalues of the Laplacian matrix (3, 1, 1, 0) One isolated vertex and two pendent vertices 29

<latexit sha1_base64="carub333yfojvhul+htsh+ofpq8=">aaacthicbzblswmxemez9v1fvy9egkxxvdyi6ewoevgoyb/qxuo2o63bbhzjsmjz+gg9epdmp/diqrhbdltqfqwm+fobmwtydxlbtxhdr6c0nt0zoze/uf5cwl5zraytn3wckgynfotytqoqqxajdconghaigeabgfzwftqqt25aar7lszniwi9ox/iez9ry1k2wy3yevqd6xgzbri3it0ps4p08sx563hdwpwgzagsyoxgbgx5f161u3zqbb/4rscgqqijzbuxbc2owriane1trdnet42dugc4edmteqigh7jr2owolpbfop8vngojts0lci5vnaxbojycygmk9ialbare80r9ri/hfrzoa3qgfczmkbiqbp9rlbtyxhjmlq66agtgwgjlf7a6yxvffmbh+l60j5pex/4rmxo24nxkxx62ffhbmo020hxyrqqeojs7qowoghu7qe3pbr8698+y8oe/j1pjtzgygh1ga/qbbxkcu</latexit> <latexit sha1_base64="carub333yfojvhul+htsh+ofpq8=">aaacthicbzblswmxemez9v1fvy9egkxxvdyi6ewoevgoyb/qxuo2o63bbhzjsmjz+gg9epdmp/diqrhbdltqfqwm+fobmwtydxlbtxhdr6c0nt0zoze/uf5cwl5zraytn3wckgynfotytqoqqxajdconghaigeabgfzwftqqt25aar7lszniwi9ox/iez9ry1k2wy3yevqd6xgzbri3it0ps4p08sx563hdwpwgzagsyoxgbgx5f161u3zqbb/4rscgqqijzbuxbc2owriane1trdnet42dugc4edmteqigh7jr2owolpbfop8vngojts0lci5vnaxbojycygmk9ialbare80r9ri/hfrzoa3qgfczmkbiqbp9rlbtyxhjmlq66agtgwgjlf7a6yxvffmbh+l60j5pex/4rmxo24nxkxx62ffhbmo020hxyrqqeojs7qowoghu7qe3pbr8698+y8oe/j1pjtzgygh1ga/qbbxkcu</latexit> <latexit sha1_base64="carub333yfojvhul+htsh+ofpq8=">aaacthicbzblswmxemez9v1fvy9egkxxvdyi6ewoevgoyb/qxuo2o63bbhzjsmjz+gg9epdmp/diqrhbdltqfqwm+fobmwtydxlbtxhdr6c0nt0zoze/uf5cwl5zraytn3wckgynfotytqoqqxajdconghaigeabgfzwftqqt25aar7lszniwi9ox/iez9ry1k2wy3yevqd6xgzbri3it0ps4p08sx563hdwpwgzagsyoxgbgx5f161u3zqbb/4rscgqqijzbuxbc2owriane1trdnet42dugc4edmteqigh7jr2owolpbfop8vngojts0lci5vnaxbojycygmk9ialbare80r9ri/hfrzoa3qgfczmkbiqbp9rlbtyxhjmlq66agtgwgjlf7a6yxvffmbh+l60j5pex/4rmxo24nxkxx62ffhbmo020hxyrqqeojs7qowoghu7qe3pbr8698+y8oe/j1pjtzgygh1ga/qbbxkcu</latexit> <latexit sha1_base64="hp+6lruf2d3tzaldqaqqvekmxyw=">aaab2xicbzdnsgmxfixv1l86vq1rn8eiucozbnqpuhfzwbzco5rm5k4bmskmyr2hdh0bf25efc93vo3pz0jbdwq+zknivscullqubn9ebwd3b/+gfugfnfzjk9nmo2fz0gjsilzl5jnmfpxu2cvjcp8lgzylffbj6f0i77+gstlxtzqrmmr4wmtuck7o6oyaraadlmw2ivxdc9yanb+gss7kddujxa0dhefbucunsafw7g9liwuxuz7ggupnm7rrtrxzzi6dk7a0n+5oykv394ukz9bostjdzdhn7ga2mp/lbiwlt1eldvesarh6kc0vo5wtdmajnchizrxwyasblykjn1yqa8z3hysbg29d77odbu3wmya6nmmfxeein3ahd9cblghi4bxevyn35n2suqp569lo4i+8zx84xio4</latexit> <latexit sha1_base64="bnjuhgqqtpwptzarpln0okiplhe=">aaacqxicbzbnswmxeizn/az1q3r1ehtfu0m86evqvhisyfxolpjnp21onrskwbes/yfephjzv3jxoihgulathwndxp6zsszvlcpphaupwdt0zozcfgmhvli0vljawvu6selmbnzfohjzfxglsmqso+kuxqugerwpviz6j6p65tuakxn97gypnmpe1bijbxcetsrimbysmmku1hkuc2fkzzbqslmkk84w/al0e7ajwcy7xioo29/xtspbteqlih8fg4stgeetvbkp24niytrokg5tg9hunxnunbqkh+uws5hy0eddbhipeyy2mrdmdmm2j23ssyxp7uhbjydyhls7icpf6rfs2d+1efyv1shc56czs51mdrx4ekitkeismnkwtkvb4dtacy6m9lss0eogc+f9l3st2o8v/xuxe1vgq+ymqgk2ybn2gce+hmep1kaoam7hez7hjbglnolxd7umgrfv6/ajgrd3lokmdg==</latexit> <latexit sha1_base64="bnjuhgqqtpwptzarpln0okiplhe=">aaacqxicbzbnswmxeizn/az1q3r1ehtfu0m86evqvhisyfxolpjnp21onrskwbes/yfephjzv3jxoihgulathwndxp6zsszvlcpphaupwdt0zozcfgmhvli0vljawvu6selmbnzfohjzfxglsmqso+kuxqugerwpviz6j6p65tuakxn97gypnmpe1bijbxcetsrimbysmmku1hkuc2fkzzbqslmkk84w/al0e7ajwcy7xioo29/xtspbteqlih8fg4stgeetvbkp24niytrokg5tg9hunxnunbqkh+uws5hy0eddbhipeyy2mrdmdmm2j23ssyxp7uhbjydyhls7icpf6rfs2d+1efyv1shc56czs51mdrx4ekitkeismnkwtkvb4dtacy6m9lss0eogc+f9l3st2o8v/xuxe1vgq+ymqgk2ybn2gce+hmep1kaoam7hez7hjbglnolxd7umgrfv6/ajgrd3lokmdg==</latexit> <latexit sha1_base64="w5pnd980xs2d1eagrsirntfe8co=">aaacthicbzblswmxemez9v1fvy9egkxxvbivehf8xdxwsa/olpjnpzwyzs5jvixlp6axd978ff48kckyblet1oehf34zk0z+qsyfsyq8eywz2bn5hcwl4vlk6tp6awozbqjec6jxsea6gtaduiiowwelngmnlawknikb81g9cqvaiehd2uem7zd1legjzqxdnri/xcfyd6avvbqezgpxn8qe72vjs9p3vwh5anqc0mmofaru9+e6tqlmkiqlpc1olsooj2qn9oh3i56eocyxzjgwjbftp0xbwsumi35iigb8hvwh5ariizh2mpkxxluodhev0i6vxrmdnehzamwgdfynw/da/k2n4h+1vmj7r+1uqdixopj4ov4isy3wyfncfrq4lqmngnfc7yr5ndomw+d/0zla/355wtqpkpru6cupn5zldiyibbsd9hffh+gexaaqqigo7tezekvv3op34r17h+pwgpfpbkffuzj/bfocpyo=</latexit> <latexit sha1_base64="carub333yfojvhul+htsh+ofpq8=">aaacthicbzblswmxemez9v1fvy9egkxxvdyi6ewoevgoyb/qxuo2o63bbhzjsmjz+gg9epdmp/diqrhbdltqfqwm+fobmwtydxlbtxhdr6c0nt0zoze/uf5cwl5zraytn3wckgynfotytqoqqxajdconghaigeabgfzwftqqt25aar7lszniwi9ox/iez9ry1k2wy3yevqd6xgzbri3it0ps4p08sx563hdwpwgzagsyoxgbgx5f161u3zqbb/4rscgqqijzbuxbc2owriane1trdnet42dugc4edmteqigh7jr2owolpbfop8vngojts0lci5vnaxbojycygmk9ialbare80r9ri/hfrzoa3qgfczmkbiqbp9rlbtyxhjmlq66agtgwgjlf7a6yxvffmbh+l60j5pex/4rmxo24nxkxx62ffhbmo020hxyrqqeojs7qowoghu7qe3pbr8698+y8oe/j1pjtzgygh1ga/qbbxkcu</latexit> <latexit sha1_base64="carub333yfojvhul+htsh+ofpq8=">aaacthicbzblswmxemez9v1fvy9egkxxvdyi6ewoevgoyb/qxuo2o63bbhzjsmjz+gg9epdmp/diqrhbdltqfqwm+fobmwtydxlbtxhdr6c0nt0zoze/uf5cwl5zraytn3wckgynfotytqoqqxajdconghaigeabgfzwftqqt25aar7lszniwi9ox/iez9ry1k2wy3yevqd6xgzbri3it0ps4p08sx563hdwpwgzagsyoxgbgx5f161u3zqbb/4rscgqqijzbuxbc2owriane1trdnet42dugc4edmteqigh7jr2owolpbfop8vngojts0lci5vnaxbojycygmk9ialbare80r9ri/hfrzoa3qgfczmkbiqbp9rlbtyxhjmlq66agtgwgjlf7a6yxvffmbh+l60j5pex/4rmxo24nxkxx62ffhbmo020hxyrqqeojs7qowoghu7qe3pbr8698+y8oe/j1pjtzgygh1ga/qbbxkcu</latexit> <latexit sha1_base64="carub333yfojvhul+htsh+ofpq8=">aaacthicbzblswmxemez9v1fvy9egkxxvdyi6ewoevgoyb/qxuo2o63bbhzjsmjz+gg9epdmp/diqrhbdltqfqwm+fobmwtydxlbtxhdr6c0nt0zoze/uf5cwl5zraytn3wckgynfotytqoqqxajdconghaigeabgfzwftqqt25aar7lszniwi9ox/iez9ry1k2wy3yevqd6xgzbri3it0ps4p08sx563hdwpwgzagsyoxgbgx5f161u3zqbb/4rscgqqijzbuxbc2owriane1trdnet42dugc4edmteqigh7jr2owolpbfop8vngojts0lci5vnaxbojycygmk9ialbare80r9ri/hfrzoa3qgfczmkbiqbp9rlbtyxhjmlq66agtgwgjlf7a6yxvffmbh+l60j5pex/4rmxo24nxkxx62ffhbmo020hxyrqqeojs7qowoghu7qe3pbr8698+y8oe/j1pjtzgygh1ga/qbbxkcu</latexit> <latexit sha1_base64="carub333yfojvhul+htsh+ofpq8=">aaacthicbzblswmxemez9v1fvy9egkxxvdyi6ewoevgoyb/qxuo2o63bbhzjsmjz+gg9epdmp/diqrhbdltqfqwm+fobmwtydxlbtxhdr6c0nt0zoze/uf5cwl5zraytn3wckgynfotytqoqqxajdconghaigeabgfzwftqqt25aar7lszniwi9ox/iez9ry1k2wy3yevqd6xgzbri3it0ps4p08sx563hdwpwgzagsyoxgbgx5f161u3zqbb/4rscgqqijzbuxbc2owriane1trdnet42dugc4edmteqigh7jr2owolpbfop8vngojts0lci5vnaxbojycygmk9ialbare80r9ri/hfrzoa3qgfczmkbiqbp9rlbtyxhjmlq66agtgwgjlf7a6yxvffmbh+l60j5pex/4rmxo24nxkxx62ffhbmo020hxyrqqeojs7qowoghu7qe3pbr8698+y8oe/j1pjtzgygh1ga/qbbxkcu</latexit> <latexit sha1_base64="carub333yfojvhul+htsh+ofpq8=">aaacthicbzblswmxemez9v1fvy9egkxxvdyi6ewoevgoyb/qxuo2o63bbhzjsmjz+gg9epdmp/diqrhbdltqfqwm+fobmwtydxlbtxhdr6c0nt0zoze/uf5cwl5zraytn3wckgynfotytqoqqxajdconghaigeabgfzwftqqt25aar7lszniwi9ox/iez9ry1k2wy3yevqd6xgzbri3it0ps4p08sx563hdwpwgzagsyoxgbgx5f161u3zqbb/4rscgqqijzbuxbc2owriane1trdnet42dugc4edmteqigh7jr2owolpbfop8vngojts0lci5vnaxbojycygmk9ialbare80r9ri/hfrzoa3qgfczmkbiqbp9rlbtyxhjmlq66agtgwgjlf7a6yxvffmbh+l60j5pex/4rmxo24nxkxx62ffhbmo020hxyrqqeojs7qowoghu7qe3pbr8698+y8oe/j1pjtzgygh1ga/qbbxkcu</latexit> <latexit sha1_base64="carub333yfojvhul+htsh+ofpq8=">aaacthicbzblswmxemez9v1fvy9egkxxvdyi6ewoevgoyb/qxuo2o63bbhzjsmjz+gg9epdmp/diqrhbdltqfqwm+fobmwtydxlbtxhdr6c0nt0zoze/uf5cwl5zraytn3wckgynfotytqoqqxajdconghaigeabgfzwftqqt25aar7lszniwi9ox/iez9ry1k2wy3yevqd6xgzbri3it0ps4p08sx563hdwpwgzagsyoxgbgx5f161u3zqbb/4rscgqqijzbuxbc2owriane1trdnet42dugc4edmteqigh7jr2owolpbfop8vngojts0lci5vnaxbojycygmk9ialbare80r9ri/hfrzoa3qgfczmkbiqbp9rlbtyxhjmlq66agtgwgjlf7a6yxvffmbh+l60j5pex/4rmxo24nxkxx62ffhbmo020hxyrqqeojs7qowoghu7qe3pbr8698+y8oe/j1pjtzgygh1ga/qbbxkcu</latexit> <latexit sha1_base64="xys+jdtlh65oyskhfer6en2tb7a=">aaacbnicbvdlssnafj3uv62vqetbbotqqutsbhvxdooygrgfnptjdnioth7o3agldofgx3hjqswt3+dov3gabthww71woodezu7xysevwnaxuzibx1hcki6xvlbx1jfmza0bfswsmodgipitjygmemgc4cbyk5ambj5gte/2yuw375lupaqvyrgznyd9kpucetbs19yt4i66k5dwrofyyuror8ehxbnsva0mejbyosmjhi2u+dnprtqjwahuekxathwdmxijnao2knusxwjcb0mfttunsccum2z3jpc+vnryj6tuehcm/t5isadumpd0zebgoka9sfif107ap3vthsyjsjbohvitgshc41bwj0tgqqw1ivry/vdmb0qscjq6kg7bnj55lji16lnvujou18/znipob+2hcrlrcaqjs9radqloat2hf/rqpbrpxpvxphktgpnonvod4+mbbtiwba==</latexit> <latexit sha1_base64="xxavqn/qwpfxnofue80g0cmwka4=">aaacuhicbvhlsgmxfl1tx7w+rl26crbftctmexqjig5cukhgq9apjzpe1mammyqzsqz9rdfu/a43lhrnpy0+6ovlts45ez2eiedaen6lu5ibx1hcki6xvlbx1jfcza2mjlpfsmfieavbkgouxgldccpwnlfio1dgtxh/ptjvhlbphstrm0iwhdg+5d3oqlfux+1fkhmshnjnmgsjahr/hbkf7bppdmumgicffbclklnpbez74ckly6dsfm/zccte1culzaj/asowqxrhfq66musjliyjqnxl9xltzqgynakclojuy0lzpe1jy0jji9ttla9kspys0yw9wnmwhutszxuzjbqerkf12gve6b/aipxpa6wmd9zouexsg5knd+qlgpiyjnilxa6qgtgwgdlf7v0ju6okmmp/ogrd8p8+ery0a1xfq/pxh+xts0kcrdibxtgah47gfc6gdg1g8asv8a4fzrpz5nwwnlf1osi2/kpc6qtsykca</latexit> <latexit sha1_base64="xxavqn/qwpfxnofue80g0cmwka4=">aaacuhicbvhlsgmxfl1tx7w+rl26crbftctmexqjig5cukhgq9apjzpe1mammyqzsqz9rdfu/a43lhrnpy0+6ovlts45ez2eiedaen6lu5ibx1hcki6xvlbx1jfcza2mjlpfsmfieavbkgouxgldccpwnlfio1dgtxh/ptjvhlbphstrm0iwhdg+5d3oqlfux+1fkhmshnjnmgsjahr/hbkf7bppdmumgicffbclklnpbez74ckly6dsfm/zccte1culzaj/asowqxrhfq66musjliyjqnxl9xltzqgynakclojuy0lzpe1jy0jji9ttla9kspys0yw9wnmwhutszxuzjbqerkf12gve6b/aipxpa6wmd9zouexsg5knd+qlgpiyjnilxa6qgtgwgdlf7v0ju6okmmp/ogrd8p8+ery0a1xfq/pxh+xts0kcrdibxtgah47gfc6gdg1g8asv8a4fzrpz5nwwnlf1osi2/kpc6qtsykca</latexit> <latexit sha1_base64="xxavqn/qwpfxnofue80g0cmwka4=">aaacuhicbvhlsgmxfl1tx7w+rl26crbftctmexqjig5cukhgq9apjzpe1mammyqzsqz9rdfu/a43lhrnpy0+6ovlts45ez2eiedaen6lu5ibx1hcki6xvlbx1jfcza2mjlpfsmfieavbkgouxgldccpwnlfio1dgtxh/ptjvhlbphstrm0iwhdg+5d3oqlfux+1fkhmshnjnmgsjahr/hbkf7bppdmumgicffbclklnpbez74ckly6dsfm/zccte1culzaj/asowqxrhfq66musjliyjqnxl9xltzqgynakclojuy0lzpe1jy0jji9ttla9kspys0yw9wnmwhutszxuzjbqerkf12gve6b/aipxpa6wmd9zouexsg5knd+qlgpiyjnilxa6qgtgwgdlf7v0ju6okmmp/ogrd8p8+ery0a1xfq/pxh+xts0kcrdibxtgah47gfc6gdg1g8asv8a4fzrpz5nwwnlf1osi2/kpc6qtsykca</latexit> <latexit sha1_base64="xxavqn/qwpfxnofue80g0cmwka4=">aaacuhicbvhlsgmxfl1tx7w+rl26crbftctmexqjig5cukhgq9apjzpe1mammyqzsqz9rdfu/a43lhrnpy0+6ovlts45ez2eiedaen6lu5ibx1hcki6xvlbx1jfcza2mjlpfsmfieavbkgouxgldccpwnlfio1dgtxh/ptjvhlbphstrm0iwhdg+5d3oqlfux+1fkhmshnjnmgsjahr/hbkf7bppdmumgicffbclklnpbez74ckly6dsfm/zccte1culzaj/asowqxrhfq66musjliyjqnxl9xltzqgynakclojuy0lzpe1jy0jji9ttla9kspys0yw9wnmwhutszxuzjbqerkf12gve6b/aipxpa6wmd9zouexsg5knd+qlgpiyjnilxa6qgtgwgdlf7v0ju6okmmp/ogrd8p8+ery0a1xfq/pxh+xts0kcrdibxtgah47gfc6gdg1g8asv8a4fzrpz5nwwnlf1osi2/kpc6qtsykca</latexit> <latexit sha1_base64="x7ulc+u1qtnmk88yvbshzn/ptrq=">aaab83icbvbns8naej3ur1q/qh69lbahgpskcuqt6mvjbwmlbsib7azdutne3u2hhp4olx5uvppnvplv3ky5aotjbh7vzbczz485u9q2v63cyura+kzxs7s1vbo7v94/efrrigl1scqj2faxopwj6mqmow3hkulq57tlj25nfmtmpwkrenctmhohhggwmik1kbwqoj9dtlb2aa9cswt2brrmnjxuieezv/7q9ioshfrowrfshceotzdiqrnhdfrqjorgmizwghymftikykuzo6foxch9fetstnaou39vpdhuahl6zjleeqgwvzn4n9djdhdlpuzeiaaczb8keo50hgyjod6tlgg+mqqtycytiayxxesbneombgfxy8verdeua/b9ravxk6drhcm4hio4caknuimmuedgcz7hfd6ssfvivvsf89gcle8cwh9ynz+ugi7p</latexit> Example 6.1 Alternative adjacency and Laplacian matrices are A = 2 3 0 0 1 0 60 0 0 0 7 41 0 0 15 0 0 1 0 L = 2 6 4 3 1 0 1 0 0 0 0 0 7 1 0 2 15 0 0 1 1 Eigenvalues of A and L are ( p 2, 0, 0, p 2) (3, 1, 1, 0) 30

Example 6.2 The bipartite graph 4 5 6 1 2 3 Figure 2.4 Bipartite Graph 31

<latexit sha1_base64="uiq7n4v+vb4qn7doovuctgcxtye=">aaac+hicdzjnb9naeibxlh8lfdsfi5creagcinytussbqyilbw5firrshexrzthzdb22dseviiv/hespgljyu7jxb1i7pk2tmtksxs37znqzu0mhlspo/wthxq3bd+5u3uvcf/dw0vz3+/exl5dw4kdmorcnixcolcebkdj4ulguwalxodl9x/vhz2idys1nmhc4ystuqfrjqb403g62psjbidwmtnobjuiep8puwloxx1syiuxraezbc+btvoqulji+rgprdw3hsilugs1j14jlxqvt/onccns3mnwli/g3kppjoyjevk1n9hlc7fe+bwlwrdskhmvjanz9hu9ywwzosgrh3ddiby38rqskxkunlh0wqp6kkq69ncjdn6qam1vac1+zqjpbn4agqs53vcjzbp4lnswezdyqvxdv8oylpqejspmijdty4knpqyfyqj8btjrfsxruhzbw+x8fornwspkppempivodev0mdvtv+vzt697hu/y0ntlt9oztsijts0p2gr2xazpbwfa1+bz8d6vwppwr/rxaw6dteckurfjrl5xfwt8=</latexit> <latexit sha1_base64="f5wptdrwxa28ujjvn9zkp7hsrku=">aaac7nicfvlbbhmxepvuwyih0lqcuywauzvl5k0qaqekfg5wlbjpk2wjyovmjla93swerypw+qkupqdiyvdw429wtlta0tanjfq0743hm3zsaowi899bulk6du/++opww41hjzfbw9snli+txj7mdw7peufqk4m9uqtxrlaosktjaxl+dq6ffkbrvg4+0rtaqsbgrqvkcvkp4vawdjr8b68h1pjsxgtqxamolametwi6q2snwambcnj9g9fvxpgdoof/gpgyw3xvzvhiio4yllbhcduemlezhmrwjsf0fnju8c6vabdj1jaoa3a8bp+kr7ksmzqktxcuh/gcbv5uulljrbwxdgshz8uy+54akaebvpv7zeczz4wgza0pq1bnb1zuinnumixemqmauevtnlym9utkxw4qzyqs0mjlrmmpgxkypz6mlevjeuqjkfb5u4kcccsk+s/s8kuifke+txr73vdd/uggc/im2cy6e8p22b6l2at2yn6zy9zjmtdbl+br8c38ff6e38mfl9ywagqesh8q/vwdaeq/tg==</latexit> 4 5 6 1 2 3 Figure 2.4 Bipartite Graph Example 6.2 The bipartite graph The adjacency metric The Laplacian L = 0 B @ A G = 0 B @ 0 0 0 1 1 1 0 0 0 1 1 0 0 0 0 0 1 0 1 1 0 0 0 0 1 1 1 0 0 0 1 0 0 0 0 0 3 0 0 1 1 1 0 2 0 1 1 0 0 0 1 0 1 0 1 1 0 2 0 0 1 1 1 0 3 0 1 0 0 0 0 1 1 C A 1 C A 32

<latexit sha1_base64="pvyggdenq5wcprfrm8comekf++c=">aaacy3icdzhbt9swfmadwkb0gwv23mwiyoidvyiqdq6tqu3cjdr10gpitvu57ktr1xei+wwrzt3uh9yn8/6ruwk2sry+ydgn7/1sv/hfmrqgff/bctc2x2xtn3aal1/t7r329g+utzprdn2eyltfrsyafar6kfdcbaabjzgem2j2avg/uqntrkq+yzhbmgetjwlbgdpo5p2+oh9okchg4yatfeyweapkwrnixvjk87pgaye+o6fb6icm/wfvteatamvuk8bj6nmgpv62d2pcn09dlsztpbl5lb/tv6lrjqhni9tqjbxf4tjleqikuwtgdai/w6e9fqwxmg+guygm8rmbwmbaxriww7iaxpwe2wrm41tbpzbw6emdjuumkzlikgndqvmtlcknaomc4/nhkvswiyi+vcjojcwulizlx0idr1lyw7gwtlfkp0wzjnb+tfsiweovr5v+wfui7x/ttlof69doklfkkbytglwnxxjjeqrpuppzsz17p3c/uoh+d38sudep97wh/8n9+qc3tbom</latexit> 6 4 7 5 Example 6.3 A complete graph 5 4 1 3 2 The diagonal degree matrix is D = 4xI where I is a 5x5 identity matrix The Laplacian is L = 0 B @ 4 1 1 1 1 1 4 1 1 1 1 1 4 1 1 1 1 1 4 1 1 1 1 1 4 1 C A 33

Example 6.4 A regular graph with D = 2xI where I is a 4x4 identity matrix 4 1 3 2 34

Graphs can be used to efficiently compute different functions of data represent data identify which vertices-data are significant reduce dimensionality and only focus on important vertices-data 35

Defining a suitable centrality metric (or index of significance) is important Centrality Closeness Betweenness Degree Eigenvector Katz PageRank 36

Degree centrality The degree vector d = Ae where A is the adjacency matrix of the graph and e is the all 1 vector. Degree'Centrality' CD = 0.16 CD = 1.0 4+4+4+4+4/5*4' 1+0+0+0+1/4*3=1/6' 1+1+0+1+0+1+1/6*5=5/30' ' 37

Degree centrality For directed graphs In degree centrality Out degree centrality 38

Degree centrality For directed graphs In degree centrality Out degree centrality 39

Degree centrality For directed graphs In degree centrality Out degree centrality 40

Eigenvector centrality Identifying important vertices in a large network is critical problem with numerous applications. Degree'Centrality' A vertex is important if its adjacent vertices are important CD = 0.16 CD = 1.0 4+4+4+4+4/5*4' 1+0+0+0+1/4*3=1/6' 1+1+0+1+0+1+1/6*5=5/30' ' 41

Eigenvector centrality Identifying important vertices in a large network is critical problem with numerous applications. Degree'Centrality' A vertex is important if its adjacent vertices are important CD = 0.16 CD = 1.0 which one is more important? 4+4+4+4+4/5*4' 1+0+0+0+1/4*3=1/6' 1+1+0+1+0+1+1/6*5=5/30' ' 42

<latexit sha1_base64="cb/yf6ymhrl/nadclzc3qyrgc80=">aaacl3icbvdnsgmxgmzwv1r/vj16crbbu9mkob6eooiepijrc92yzno0tzvnhirbkms+khcfrs8kkl59c7ptilodcawz8/hlm0awqrtjvfifufmfxaxicmlldw19w97culnrldfxccqi2qyqioxy4mqqgwkksvaymniihuez3xgrqwjeb/vykhaiepx2kubasl59eeeni5+m0bmyejqcnopdpxlaj3kyebgxej1m/iq3sohpndgaye/o4fv37bjtcsaaf0k1j2wqo+7bj14nwnfiumymkdwqokk3eyq1xyykjs9wrca8rd3smpsjkkh2mjk4hxtg6cbujm3jgk7unxmjcpuah4fjhkj31ayxif95rvh3j9sj5slwhoppom7mockmaw92qcrys7ehcetq/gpxh0metem4zeqozp78l7ghlzokc3nyrp3lbrtbdtgf+6akjkanxie6caeg9+ajvii368f6tt6tj2m0youz2+axrm8vroeplq==</latexit> Eigenvector centrality Identifying important vertices in a large network is critical problem with numerous applications. A vertex is important if its adjacent vertices are important Centrality is proportional to the centrality of adjacent vertices E vi / X j2n i E vj = X A system of equations with n unknowns j a ij E vj 43

<latexit sha1_base64="p8uvqdbzqejvr9e6+xnfnbvogaw=">aaadkxicfzjnbxmxeia9w6bt+erhygvebcqxyfshqq9iaq5wlbkhlbjr5hvme6te78qerypw+t1c+cu99mdxlt9sj9lw6ayw0kiv5n08tsdocq0ccf47cldu3b6zvbpbunvv/ooh7b1hx1xewol9mevcnitcovyg+6ri40lhuwsjxupk9p3cp/6k1qncfkzzgcnmtixklrtks6o9opd29aheqkwxpf0wlcnockjmjawvs3klvzgvtqaozyhyyrszxfevc+k3uxvm3b/onaamgp0l2uj4h+zqx8apb7ozh8vjobnxc4hyqsmuxozahd7ly4bnedwiw+o4grxp43euywwnss2cg0s8okhvskpqnlfi0meh5kmy4mblizj0w2r51hn45itjshpr0xasq+srkpe5n8sst2acpq7plyo3eyos0tfdspmijdrytvfaaqacfv8gxsqijd3zqkir/flbtouvkvzvavkhrm0rb4r+qfewyz+97pte1dpyyu/yu7bpivak9dhhdst6tabfgrpgr/az/b6eh7/cpys0doo1j9m1cp9eaix2yr0=</latexit> <latexit sha1_base64="w0rwl5ut/zgnrm+fy6fdhfjih7m=">aaacanicbvdlsgmxfl3js9bxqdvdbivgqsyioc6eqogukzi20a5djpnpqzmpkkyhdau3/oobfypu/qp3/o1powttpra4oedcknv8ldoplovbmjtfwfxalq2uv9fwnzbnre0hmwscuickpbfnh0vkwuwdxrsnzvrqhpmcnvze1chv9kmqlinv1sclboq7mqszwuplnrnb5jocyhtt5f0hokfowruzxdyzylwtmdassqtsgqj1z/xqbwnjihorwrguldtklztjorjhdfhuz5kmmprwh7y0jxfepzupdxiia60ekeyeprfcy/x3ri4jkqerr5mrvl057y3e/7xwpsjtn2dxmikak8ldycarstcoebqwqynia00weuz/fzeufpgoxvtzl2bprzxlnkpqwdw6o67ulos2srah+3ainpxadw6hdg4qeirneiu348l4md6nj0l0zihmduapjm8fnm2wjw==</latexit> <latexit sha1_base64="cb/yf6ymhrl/nadclzc3qyrgc80=">aaacl3icbvdnsgmxgmzwv1r/vj16crbbu9mkob6eooiepijrc92yzno0tzvnhirbkms+khcfrs8kkl59c7ptilodcawz8/hlm0awqrtjvfifufmfxaxicmlldw19w97culnrldfxccqi2qyqioxy4mqqgwkksvaymniihuez3xgrqwjeb/vykhaiepx2kubasl59eeeni5+m0bmyejqcnopdpxlaj3kyebgxej1m/iq3sohpndgaye/o4fv37bjtcsaaf0k1j2wqo+7bj14nwnfiumymkdwqokk3eyq1xyykjs9wrca8rd3smpsjkkh2mjk4hxtg6cbujm3jgk7unxmjcpuah4fjhkj31ayxif95rvh3j9sj5slwhoppom7mockmaw92qcrys7ehcetq/gpxh0metem4zeqozp78l7ghlzokc3nyrp3lbrtbdtgf+6akjkanxie6caeg9+ajvii368f6tt6tj2m0youz2+axrm8vroeplq==</latexit> E vi / X j2n i E vj = X j a ij E vj Eigenvector centrality E v = A G E v The eigenvector of the adjacency matrix A G = 0 B @ 0 1 0 0 0 0 1 1 0 1 0 0 0 1 0 1 0 1 0 0 0 0 0 1 0 1 1 0 0 0 0 1 0 1 0 0 0 0 1 1 0 0 1 C A 1 1 0 0 0 0 0 44

<latexit sha1_base64="p8uvqdbzqejvr9e6+xnfnbvogaw=">aaadkxicfzjnbxmxeia9w6bt+erhygvebcqxyfshqq9iaq5wlbkhlbjr5hvme6te78qerypw+t1c+cu99mdxlt9sj9lw6ayw0kiv5n08tsdocq0ccf47cldu3b6zvbpbunvv/ooh7b1hx1xewol9mevcnitcovyg+6ri40lhuwsjxupk9p3cp/6k1qncfkzzgcnmtixklrtks6o9opd29aheqkwxpf0wlcnockjmjawvs3klvzgvtqaozyhyyrszxfevc+k3uxvm3b/onaamgp0l2uj4h+zqx8apb7ozh8vjobnxc4hyqsmuxozahd7ly4bnedwiw+o4grxp43euywwnss2cg0s8okhvskpqnlfi0meh5kmy4mblizj0w2r51hn45itjshpr0xasq+srkpe5n8sst2acpq7plyo3eyos0tfdspmijdrytvfaaqacfv8gxsqijd3zqkir/flbtouvkvzvavkhrm0rb4r+qfewyz+97pte1dpyyu/yu7bpivak9dhhdst6tabfgrpgr/az/b6eh7/cpys0doo1j9m1cp9eaix2yr0=</latexit> <latexit sha1_base64="w0rwl5ut/zgnrm+fy6fdhfjih7m=">aaacanicbvdlsgmxfl3js9bxqdvdbivgqsyioc6eqogukzi20a5djpnpqzmpkkyhdau3/oobfypu/qp3/o1powttpra4oedcknv8ldoplovbmjtfwfxalq2uv9fwnzbnre0hmwscuickpbfnh0vkwuwdxrsnzvrqhpmcnvze1chv9kmqlinv1sclboq7mqszwuplnrnb5jocyhtt5f0hokfowruzxdyzylwtmdassqtsgqj1z/xqbwnjihorwrguldtklztjorjhdfhuz5kmmprwh7y0jxfepzupdxiia60ekeyeprfcy/x3ri4jkqerr5mrvl057y3e/7xwpsjtn2dxmikak8ldycarstcoebqwqynia00weuz/fzeufpgoxvtzl2bprzxlnkpqwdw6o67ulos2srah+3ainpxadw6hdg4qeirneiu348l4md6nj0l0zihmduapjm8fnm2wjw==</latexit> <latexit sha1_base64="cb/yf6ymhrl/nadclzc3qyrgc80=">aaacl3icbvdnsgmxgmzwv1r/vj16crbbu9mkob6eooiepijrc92yzno0tzvnhirbkms+khcfrs8kkl59c7ptilodcawz8/hlm0awqrtjvfifufmfxaxicmlldw19w97culnrldfxccqi2qyqioxy4mqqgwkksvaymniihuez3xgrqwjeb/vykhaiepx2kubasl59eeeni5+m0bmyejqcnopdpxlaj3kyebgxej1m/iq3sohpndgaye/o4fv37bjtcsaaf0k1j2wqo+7bj14nwnfiumymkdwqokk3eyq1xyykjs9wrca8rd3smpsjkkh2mjk4hxtg6cbujm3jgk7unxmjcpuah4fjhkj31ayxif95rvh3j9sj5slwhoppom7mockmaw92qcrys7ehcetq/gpxh0metem4zeqozp78l7ghlzokc3nyrp3lbrtbdtgf+6akjkanxie6caeg9+ajvii368f6tt6tj2m0youz2+axrm8vroeplq==</latexit> E vi / X j2n i E vj = X j a ij E vj Eigenvector centrality E v = A G E v The eigenvector of the adjacency matrix A G = 0 B @ 0 1 0 0 0 0 1 1 0 1 0 0 0 1 0 1 0 1 0 0 0 0 0 1 0 1 1 0 0 0 0 1 0 1 0 0 0 0 1 1 0 0 1 C A 1 1 0 0 0 0 0 45

Intuition starts with degree centrality Degree vector Incorporating the degree of the neighbors 46 X = 0 B B B B B B B B @ 2 3 2 3 2 2 2 1 C C C C C C C C A <latexit sha1_base64="pmazh83mzka9fqedbef0pork0iy=">aaacfxicdvhbsgmxem2u93qr+uhltcjeknsqqa9c0rcffawwuqvk09k2njtdklmhlp0mf8w3v8ux03yv6+va4mw5m5lkjkikmoh5b447mzs3v7c4vfhewv1bl25sppo41rzqpjaxbgtmgbqk6ihqqiprwkjawlpqvxn5t8+gjyjvaw4saewsq0qooemrtysvdxpffqkhhhtogh4axaeypjubddm+zgvkq9t3v4lt70gv/m9npu3xfdye1cnbuv+lbg8p28wsv/bgol9jjsclkuouxxz1ozfpi1dijtomwfesbnlbuxajw4kfgkgy77munc1vlaltysbjg9i9q3roggt7fnkx+r0iy5exgyiwmrhdnvnpjcs/vgak4uureypjersfnaptstgmo13qjtdauq4syvwl+1bke0wzjnzjbtueys8v/yb1avmy7n2flwrx+tqwytbzjqekqs5jjdyso1innlw7o86rc+wsd989ccutvnfja7bifnzzd+brqyy=</latexit> <latexit sha1_base64="pmazh83mzka9fqedbef0pork0iy=">aaacfxicdvhbsgmxem2u93qr+uhltcjeknsqqa9c0rcffawwuqvk09k2njtdklmhlp0mf8w3v8ux03yv6+va4mw5m5lkjkikmoh5b447mzs3v7c4vfhewv1bl25sppo41rzqpjaxbgtmgbqk6ihqqiprwkjawlpqvxn5t8+gjyjvaw4saewsq0qooemrtysvdxpffqkhhhtogh4axaeypjubddm+zgvkq9t3v4lt70gv/m9npu3xfdye1cnbuv+lbg8p28wsv/bgol9jjsclkuouxxz1ozfpi1dijtomwfesbnlbuxajw4kfgkgy77munc1vlaltysbjg9i9q3roggt7fnkx+r0iy5exgyiwmrhdnvnpjcs/vgak4uureypjersfnaptstgmo13qjtdauq4syvwl+1bke0wzjnzjbtueys8v/yb1avmy7n2flwrx+tqwytbzjqekqs5jjdyso1innlw7o86rc+wsd989ccutvnfja7bifnzzd+brqyy=</latexit> <latexit sha1_base64="pmazh83mzka9fqedbef0pork0iy=">aaacfxicdvhbsgmxem2u93qr+uhltcjeknsqqa9c0rcffawwuqvk09k2njtdklmhlp0mf8w3v8ux03yv6+va4mw5m5lkjkikmoh5b447mzs3v7c4vfhewv1bl25sppo41rzqpjaxbgtmgbqk6ihqqiprwkjawlpqvxn5t8+gjyjvaw4saewsq0qooemrtysvdxpffqkhhhtogh4axaeypjubddm+zgvkq9t3v4lt70gv/m9npu3xfdye1cnbuv+lbg8p28wsv/bgol9jjsclkuouxxz1ozfpi1dijtomwfesbnlbuxajw4kfgkgy77munc1vlaltysbjg9i9q3roggt7fnkx+r0iy5exgyiwmrhdnvnpjcs/vgak4uureypjersfnaptstgmo13qjtdauq4syvwl+1bke0wzjnzjbtueys8v/yb1avmy7n2flwrx+tqwytbzjqekqs5jjdyso1innlw7o86rc+wsd989ccutvnfja7bifnzzd+brqyy=</latexit> A G X = 0 B B B B B B B B @ 0 1 0 0 0 0 1 1 0 1 0 0 0 1 0 1 0 1 0 0 0 0 0 1 0 1 1 0 0 0 0 1 0 1 0 0 0 0 1 1 0 0 1 1 0 0 0 0 0 1 C C C C C C C C A 0 B B B B B B B B @ 2 3 2 3 2 2 2 1 C C C C C C C C A = 0 B B B B B B B B @ 5 6 6 6 5 5 5 1 C C C C C C C C A <latexit sha1_base64="undxforystuarjtjjbqmapfhkjg=">aaaevhicjvrnj9mwehwtakubpqthlimq0hkpki7lx2glbq5wxctkvmqqynenrvxhiwwhqyr6j/ec+cdckhdb7jkk24qrrhrpezmep2kcpojr43m/go7bvhp33sh91oohjw4ft4+efndjphj2wsisnqiprsel9g03agepqhqhai/d+acvfvkdleaj/gywky5iopu84owamxofofmp488wgdmibebmuavrc0kccpltpehimbonlqsqwiox4fv3au4hwq3ngq/zypwyxuxvshwiv4u01xep5+bg2ts3vcyda2vqtgpdifb8ojovwnu1+6darzzwsfnqg91qhvat2tpsgfznukflnm+qh53qatmuahuhgrc7xtdbg2whfhf0sgex4/zvmelyfqm0tfcth76xmpftazgtugwfmcausjmd4tcgksaor/l6k5bwwmymecxkujswzpyrchprvyhdy4ypmek6tkrehg0ze70b5vymmuhjnhdfmqctwgrfymivmimwnqbmctsrsblvlbm7icsr/pqtt4n+r/u+63193tn/wkhxqj6r5+sy+oqtosdfyaxpe+zcob/dhuu4p90/tftlbkhoo6h5sirwppwlqwoixa==</latexit> <latexit sha1_base64="undxforystuarjtjjbqmapfhkjg=">aaaevhicjvrnj9mwehwtakubpqthlimq0hkpki7lx2glbq5wxctkvmqqynenrvxhiwwhqyr6j/ec+cdckhdb7jkk24qrrhrpezmep2kcpojr43m/go7bvhp33sh91oohjw4ft4+efndjphj2wsisnqiprsel9g03agepqhqhai/d+acvfvkdleaj/gywky5iopu84owamxofofmp488wgdmibebmuavrc0kccpltpehimbonlqsqwiox4fv3au4hwq3ngq/zypwyxuxvshwiv4u01xep5+bg2ts3vcyda2vqtgpdifb8ojovwnu1+6darzzwsfnqg91qhvat2tpsgfznukflnm+qh53qatmuahuhgrc7xtdbg2whfhf0sgex4/zvmelyfqm0tfcth76xmpftazgtugwfmcausjmd4tcgksaor/l6k5bwwmymecxkujswzpyrchprvyhdy4ypmek6tkrehg0ze70b5vymmuhjnhdfmqctwgrfymivmimwnqbmctsrsblvlbm7icsr/pqtt4n+r/u+63193tn/wkhxqj6r5+sy+oqtosdfyaxpe+zcob/dhuu4p90/tftlbkhoo6h5sirwppwlqwoixa==</latexit> <latexit sha1_base64="undxforystuarjtjjbqmapfhkjg=">aaaevhicjvrnj9mwehwtakubpqthlimq0hkpki7lx2glbq5wxctkvmqqynenrvxhiwwhqyr6j/ec+cdckhdb7jkk24qrrhrpezmep2kcpojr43m/go7bvhp33sh91oohjw4ft4+efndjphj2wsisnqiprsel9g03agepqhqhai/d+acvfvkdleaj/gywky5iopu84owamxofofmp488wgdmibebmuavrc0kccpltpehimbonlqsqwiox4fv3au4hwq3ngq/zypwyxuxvshwiv4u01xep5+bg2ts3vcyda2vqtgpdifb8ojovwnu1+6darzzwsfnqg91qhvat2tpsgfznukflnm+qh53qatmuahuhgrc7xtdbg2whfhf0sgex4/zvmelyfqm0tfcth76xmpftazgtugwfmcausjmd4tcgksaor/l6k5bwwmymecxkujswzpyrchprvyhdy4ypmek6tkrehg0ze70b5vymmuhjnhdfmqctwgrfymivmimwnqbmctsrsblvlbm7icsr/pqtt4n+r/u+63193tn/wkhxqj6r5+sy+oqtosdfyaxpe+zcob/dhuu4p90/tftlbkhoo6h5sirwppwlqwoixa==</latexit>

<latexit sha1_base64="y840c6ung8/1cwkq3ko/hcq6nwo=">aaaevnicjvppa9swffztd+2yx2l33eusbhsxijv1aw+dsl127gbzc3eisvkcimqykercmpknyy79v3rzlmttbkc1e/dg8/s+vffpguwzfmyscrpl+ch2k53dp51nz1+8fnxd2/9l0lxzgpbupvoiygakudcwwkq4ydswjjjwhl1+w/lnv6cnsnvpo89gllcperhgzlrsem9lqgmxpejgvyqrtiuqmnzsvij4ohylithb7zf1srpjw/bec8c3dvvnla/rgqkmkd4m2ujyormf2hd90m0ivtsmqem5fxxqmz3zd45pw96/tr1vhx+qfzxkhvvxjcbtnr448v9cuvrtsmvd6sfuiwu/u96ftprrjls90ierwjualqchyjgbd6/dscrzbjtlkhkzpcszi9fxci5h0qlzaxnjl2wkqwcvs8cmitw7wob3rjlbcapdkotx1eqjgixgzjpikrnmz6bjlyspccpcxsejqqgst6d4elccs2xtvhxkeci0ccvndjcuhfok+yxpxq17issl0oavn8hgsh/sjz8+9k6/ltvyrw/qw3sakpqmttf3diygihu/vvvf9wp/xv8tbac7a6m3vz55jwordp8criiiiq==</latexit> <latexit sha1_base64="w0rwl5ut/zgnrm+fy6fdhfjih7m=">aaacanicbvdlsgmxfl3js9bxqdvdbivgqsyioc6eqogukzi20a5djpnpqzmpkkyhdau3/oobfypu/qp3/o1powttpra4oedcknv8ldoplovbmjtfwfxalq2uv9fwnzbnre0hmwscuickpbfnh0vkwuwdxrsnzvrqhpmcnvze1chv9kmqlinv1sclboq7mqszwuplnrnb5jocyhtt5f0hokfowruzxdyzylwtmdassqtsgqj1z/xqbwnjihorwrguldtklztjorjhdfhuz5kmmprwh7y0jxfepzupdxiia60ekeyeprfcy/x3ri4jkqerr5mrvl057y3e/7xwpsjtn2dxmikak8ldycarstcoebqwqynia00weuz/fzeufpgoxvtzl2bprzxlnkpqwdw6o67ulos2srah+3ainpxadw6hdg4qeirneiu348l4md6nj0l0zihmduapjm8fnm2wjw==</latexit> The process of adjusting the significance of a node based on the significance of neighbors can continue till the adjustment settles 0 B @ 0 1 0 0 0 0 1 1 0 1 0 0 0 1 0 1 0 1 0 0 0 0 0 1 0 1 1 0 0 0 0 1 0 1 0 0 0 0 1 1 0 0 1 1 0 0 0 0 0 Leading to an eigenvector of the matrix A 1 0 C B A @ 5 6 6 6 5 5 5 1 C A = 0 B @ 11 16 12 16 11 11 11 1 C A E v = A G E v 47

<latexit sha1_base64="xnylzn9er7rtcfqz1fnuxnjzkek=">aaaclhichvfnsynbeo0zxvezfmt14mflyrdcs5gyzbmhwq8et6jgvietqk+njmns0zn01whhyd/y13jz39iji2gufndw6r2q7uqqkfpsuha8e/7c4o+ln8srlv+ra+sb1d+b/22ag4ftkaru3efcopia2yrj4x1mkcerwrtodd717x7qwjnqwxpn2e34qmtyck5o6lufl3opcayhwpj2kzbdgofa6oibw8etqkxkgscon5sqhu/iw6opcbmf3yq058j5g+ctc9t9sgcijrwm6u+vwgvqwqzwmtrkummlrnvvp7cfijxbtujxazunikouu5wkudiphlnfjisrh2dhuc0ttn1intuj7dmld3fq3neem/v9rceta8dj5dittkm7703fr7xotngrw0id5yravd4u5woohemkoc8ncljjr7gw0vukysgnf+qwwxfdamx/+tnph9t/1yobw9rjwtmnzbbddtk+a7c/7irdsmvwzslb9freqxfmb/vh/rl/8zrqe2xnfvsa/+ofl42v1g==</latexit> The set of eigenvalues are -1.81-1.00-1.00-1.00 0.47 2.00 2.34 The eigenvector corresponding to the largest eigenvalue will have nonnegative elements since the adjacency matrix has non-negative elements ( from the Perron-Frobenius theorem) That is also the best lower rank approximation of the matrix A Eigenvalue 2.34 and the corresponding eigenvector E v = 0 B @ 0.33 0.45 0.38 0.45 0.33 0.33 0.33 1 C A 48

<latexit sha1_base64="xnylzn9er7rtcfqz1fnuxnjzkek=">aaaclhichvfnsynbeo0zxvezfmt14mflyrdcs5gyzbmhwq8et6jgvietqk+njmns0zn01whhyd/y13jz39iji2gufndw6r2q7uqqkfpsuha8e/7c4o+ln8srlv+ra+sb1d+b/22ag4ftkaru3efcopia2yrj4x1mkcerwrtodd717x7qwjnqwxpn2e34qmtyck5o6lufl3opcayhwpj2kzbdgofa6oibw8etqkxkgscon5sqhu/iw6opcbmf3yq058j5g+ctc9t9sgcijrwm6u+vwgvqwqzwmtrkummlrnvvp7cfijxbtujxazunikouu5wkudiphlnfjisrh2dhuc0ttn1intuj7dmld3fq3neem/v9rceta8dj5dittkm7703fr7xotngrw0id5yravd4u5woohemkoc8ncljjr7gw0vukysgnf+qwwxfdamx/+tnph9t/1yobw9rjwtmnzbbddtk+a7c/7irdsmvwzslb9freqxfmb/vh/rl/8zrqe2xnfvsa/+ofl42v1g==</latexit> 0.33 0.45 0.38 E v = 0 B @ 0.33 0.45 0.38 0.45 0.33 0.33 1 C A 0.33 0.33 49

Eigenvalue 2.34 and the corresponding eigenvector The average degree of vertices is 2.28 It can be shown that 2.28 < 2.34 < 3, that is, the value of the largest eigenvalue of A is between the average degree and the maximum degree of the vertices The consequence of eigenvector centrality is to only focus on critical vertices and reduce the dimensionality of the problem. 50

A graph can also capture the way joint probability distributions of all variables can be decomposed and then computed Different graphical models for inference Bayesian networks Markov random fields Factor graph 51

Example 6.5 A common motivating example Difficulty of an exam, intelligence of the student, grade in a class, student s SAT exam results, professor s letter of recommendation Denoted as D, i, g, S, l, respectively How is the dependency structure of all these variables? 52

Example 6.5 How is the dependency structure of all these variables? Intuitively Difficulty Intelligence grade SAT letter 53

Example 6.5 Difficulty Intelligence The joint probability grade SAT Lets find out how letter 54

Example 6.6 Some basic concepts for two random variables, that is, two data sets F X1, (a, b) =Pr{ apple a, apple b}, (a, b) =Pr{ apple a, apple b} = Pr{A = {w 2 (w) apple a} \ B = {w 2 (w) apple b}} = Pr{B A}Pr{A} 55

Example 6.6 Some basic concepts for two random variables, that is, two data sets F X1, (a, b) =Pr{ apple a, apple b} F X1, (a, b) =F X2 (b a)f X1 (a) = Pr{ apple a, apple b} = Pr{ apple b apple a}pr{ apple a} F X2 (b) = lim F,X (a, b) = 2 a!+1 lim Pr{ apple a, apple b} a!+1 56

If the data sets are discrete valued then probability mass functions (pmf s) are defined and we will have similar implications p X1, (a, b) =Pr{ = a, = b} p X1, (a, b) =p X2 (b a)p X1 (a) = Pr{ = a, = b} = Pr{ = b = a}pr{ = a} p X2 (b) = X a p X1, (a, b) = X a Pr{ = a, = b} 57

If the data sets were continuous valued then probability density functions (pdf s) will be defined and we will have similar implications Z b Z a F X1, (a, b) =Pr{ apple a, apple b} = 1 1 f X1, (x 1,x 2 )dx 1 dx 2 f X1, (x 1,x 2 )=f X2 (x 2 x 1 )f X1 (x 1 ) f X2 (x 2 )= Z +1 1 f X1, (x 1,x 2 )dx 1 = Z +1 1 f X2 (x 2 x 1 )f X1 (x 1 )dx 1 58

Efficient graphical models Compute joint distribution of data global function of multiple variables F X = F X1,,,X 4,X 5 Marginalize F X3 (x 3 )= lim x 1!+1 lim x 2!+1 lim x 4!+1 lim x 5!+1 F,,,X 4,X 5 59

Efficient graphical models Compute joint distribution of data global function of multiple variables f X = f X1,,,X 4,X 5 Marginalize f X3 (x 3 )= Z +1 1 Z +1 1 Z +1 1 Z +1 1 f X1,,,X 4,X 5 (x 1,x 2,x 3,x 4,x 5 )dx 1 dx 2 dx 4 dx 5 60

Efficient graphical models Compute joint distribution of data global function of multiple variables p X (x) =p X1,,,X 4,X 5 Marginalize X p (x X3 3 )= x 1 X x 2 X x 4 X x 5 p X1,,,X 4,X 5 (x 1,x 2,x 3,x 4,x 5 ) 61

Critical for inference problems The global function factorizing into local functions F X1, (a, b) =F X2 (b a)f X1 (a) f X1, (x 1,x 2 )=f X2 (x 2 x 1 )f X1 (x 1 ) p X1, (a, b) =p X2 (b a)p X1 (a) Graphical models are powerful tools in representing these expressions 62

Bayesian network directed graphs Consider three variables and their joint distribution F X1,, (x 1,x 2,x 3 )=F X3, (x 3 x 1,x 2 )F X1, (x 1,x 2 ) = F X3, (x 3 x 1,x 2 )F X2 (x 2 x 1 )F X1 (x 1 ) 63

Bayesian network directed graphs Consider three variables and their joint distribution F X1,, (x 1,x 2,x 3 )=F X3, (x 3 x 1,x 2 )F X1, (x 1,x 2 ) = F X3, (x 3 x 1,x 2 )F X2 (x 2 x 1 )F X1 (x 1 ) 64

Bayesian network directed graphs Consider three variables and their joint distribution F X1,, (x 1,x 2,x 3 )=F X3, (x 3 x 1,x 2 )F X1, (x 1,x 2 ) = F X3, (x 3 x 1,x 2 )F X2 (x 2 x 1 )F X1 (x 1 ) 65

Bayesian network directed graphs Consider three variables and their joint distribution F X1,, (x 1,x 2,x 3 )=F X3, (x 3 x 1,x 2 )F X1, (x 1,x 2 ) = F X3, (x 3 x 1,x 2 )F X2 (x 2 x 1 )F X1 (x 1 ) 66

Bayesian network directed graphs Consider three variables and their joint distribution F X1,, (x 1,x 2,x 3 )=F X3, (x 3 x 1,x 2 )F X1, (x 1,x 2 ) = F X3, (x 3 x 1,x 2 )F X2 (x 2 x 1 )F X1 (x 1 ) 67

If and are independent F X1,, (x 1,x 2,x 3 )=F X3, (x 3 x 1,x 2 )F X2 (x 2 )F X1 (x 1 ) 68

If and are independent F X1,, (x 1,x 2,x 3 )=F X3, (x 3 x 1,x 2 )F X2 (x 2 )F X1 (x 1 ) Here, are the parents of The computation of joint density Decomposed Tractable 69

An alternative order of conditioning would lead to F X1,, = F X1, F X2, = F X1, F X2 F X3 70

If and are independent This graph will not be reduced F X1,, = F X1, F X2, = F X1, F X2 F X3 71

If and are independent This graph will not be reduced Since and may not be independent conditioned on F X1,, = F X1, F X2, = F X1, F X2 F X3 72

If and are independent This graph will not be reduced Since and may not be independent conditioned on Can we construct a counter example to show? F X1,, = F X1, F X2, = F X1, F X2 F X3 73

However, if and were independent then The graph will be reduced F X1,, = F X1, F X2, = F X1, F X2 F X3 F X1,, = F X1, F X2 F X3 74

If the graph is X 4 X 5 X 6 X 7 75

If the graph is X 4 X 5 X 6 X 7 Then, F X1,,...,X 7 = F X1 F X2 F X3 F X4,, F X5, F X6 X 4 F X7 X 4,X 5 76

If the graph is Y...... X n 77

If the graph is Y...... X n Then, F X1,,...,X n,y = F Y F X1 Y F X2 Y...F Xn Y 78

The repetition could be simplified by defining a plate Y X i n 79

Graphical probabilistic model with deterministic parameters F X,Y s,. 2 = F Y n Y i=1 F Xi Y,s i, 2 Y 2 X i s i n 80

Graphical probabilistic model with deterministic parameters F X,Y s,. 2 = F Y n Y i=1 F Xi Y,s i, 2 For example, F Xi Y,s i, 2 Gaussian Y 2 X i s i n 81

Graphical probabilistic model with observed variables F X,Y s,. 2 = F Y n Y i=1 F Xi Y,s i, 2 Y 2 X i observed variables s i n 82

Graphical probabilistic model with observed variables F X,Y s,. 2 = F Y n Y i=1 F Xi Y,s i, 2 latent variable Y 2 X i s i n 83

Can we infer independence or conditional independence from Bayesian graphs? Let us investigate via a few simple examples. The joint pmf of these variables using the graph is p X1,, = p X3 p X1 p X2 84

Can we infer independence or conditional independence from Bayesian graphs? Let us investigate via a few simple examples. p X1,, = p X3 p X1 p X2 p X1, = p,, p X3 = p X1 p X2 They are independent conditioned on Node is tail-to-tail with respect to path from to 85

Can we infer independence or conditional independence from Bayesian graphs? Let us investigate via a few simple examples. The joint pmf of these variables using the graph p X1,, = p X3 p X1 p X2 and are independent conditioned on Are, independent? 86

Can we infer independence or conditional independence from Bayesian graphs? Let us investigate via a few simple examples. p X1,, = p X3 p X1 p X2 p X1, = X x 3 p X1,, = X x 3 p X3 p X1 p X2 6= p X1 p X2 They are not independent unless and as well as and are independent 87

Another example p X1,, = p X1 p X3 p X2 p X1, = X x 3 p X1,, = p X1 X x 3 p X3 p X2 6= p X1 p X2 They are not independent 88

Another example p X1,, = p X1 p X3 p X2 p X1, = p,, p X3 = p p X3 p X2 p X3 = p X1 p X2 They are independent conditioned on Node is head-to-tail with respect to path from to 89

Another example p X1,, = p X1 p X2 p X3, Are, independent? 90

Another example p X1,, = p X1 p X2 p X3, Are, independent? X X p X1,X = p 2 X1,,X = p p 3 X1 X2 x 3 Yes they are independent p X3, = p p X1 X2 91

Another example p X1,, = p X1 p X2 p X3, Are, conditioned on independent? p X1, = p,, p X3 = p p X2 p X3, p X3 6= p X1 p X2 They are not independent conditioned on Node is head-to-head with respect to path from to 92

Example 6.7 Two random variable are independent Conditioned on a third random variable then they are not. Assume X and Y are independent random binary data (that is basically a coin flip experiment). Equally likely to 0 or 1. 93

Example 6.7 Then by assumption they are independent. Define Z to be another random variable as Z = X+Y X and Y are dependent conditioned on Z = 1 P (X =1,Y =1 Z = 1) = 0 however P (X =1 Z = 1)P (Y =1 Z = 1) = 1/2 1/2 =1/4 94

Summary of and independence Conditionally independent but not independent not blocked unless the node on the path is observed Conditionally independent but not independent not blocked unless the node on the path is observed Independent but not conditionally independent blocked unless the blocking node is observed 95

Bayesian networks A tail-to-tail node or head-to-tail node leaves a path unblocked unless the node is observed (that is, the distribution is conditioned on that variable). In that case it blocks the path Conditionally independent but not independent 96

Bayesian networks A tail-to-tail node or head-to-tail node leaves a path unblocked unless the node is observed (that is, it is conditioned on that variable). In that case it blocks the path A head-to-head node blocks the path if it is unobserved If the node, and/or at least one of its descendants, is observed then the path becomes unblocked Independent but not conditionally independent 97

Bayesian networks A tail-to-tail node or head-to-tail node leaves a path unblocked unless the node is observed (that is, it is conditioned on that variable). In that case it blocks the path A head-to-head node blocks the path if it is unobserved If the node, and/or at least one of its descendants, is observed then the path becomes unblocked When the path between two nodes is blocked then the two nodes (the variables) are independent 98

These rules apply to larger networks and to sets of nodes The path between and Unblocked by X 5 Tail-to-tail X 5 Blocked by Head-to-head and are independent X 4 99

If the path between two nodes is blocked then the nodes are independent conditioned on the variable that blocked the path 100

These rules apply to larger networks and to sets of nodes The path between and Blocked by X 5 Conditioned X 5 Tail-to-tail Blocked by Head-to-head and are independent X 4 101

These rules apply to larger networks and to sets of nodes The path between and Unblocked by Head-to-head X 5 Conditioned on its descendent Unblocked byx 5 Tail-to-tail and are not independent X 4 102

In general, Bayesian networks can be represented as p X = KY k=1 p Xk p a (k) where p a (k) is the set of parent s of node k Note that Bayesian graphs do not have cycles Directed acyclic graph Invalid p X1 p X2 p X3 103

Graphical modeling for inference Bayesian networks Markov random fields Factor graphs 104

Conditional independence is often difficult to infer from directed graphs. Undirected graphs are also powerful tools Markov undirected networks Clique A group of nodes fully connected Maximal clique Cliques that can not be expanded 105

Cliques X 4 106

Cliques X 4 107

Cliques X 4 108

Maximal clique X 4 109

Maximal clique X 4 110

Not a clique X 4 111

The probability distribution can be written as F X = 1 Z Y C C(X) where C is the potential function of clique An example, X n 1 X n F X = F X1,,...,X n = F X1 F X2 F X3...F Xn X n 1 F X = 1 Z 1,2 (, ) 2,3 (, )... n 1,n (X n 1,X n ) 112

The network X n 1 X n... 1,2(, )=F X1 F X2 2,3(, )=F X3 n 1,n(X n 1,X n )=F Xn X n 1 F X = 1 Z 1,2 (, ) 2,3 (, )... n 1,n (X n 1,X n ) 113

A less obvious example F X = F X1,,,X 4,X 5 = F X1 F X2 F X3, F X4 F X5 X 5 X 4 114

<latexit sha1_base64="ogep1qiwgh7wefqptgar9csmc3a=">aaachnicbvdlsgmxfm3uv62vqks3wsk4kjnsx7gpunfzwdpcw4zm5rynzwsg5i5ysv/ejb/ixowk4er/xvsx0nydcsfn3ppknicrwqdrfjuzhcwl5zxsam5tfwnzk7+9c2fivhoo8ljguh4wa1ioqkjacfvea4sccbwgdzwq1+5bgxgrw+wn0ipyr4m24ayt5odp6n6jni9oe+ebb0yfw9gp7h//ejvqfsmvkoqe7kzw6oclbtedg84tb0okziqkn/9shjfpi2vmkhnt8nwew/zuffzcmndmdssm91ghgpyqfofpdcbzdembvulajrvdculy/e0ysmiyfhtyzohh18zwruj/tuak7bpwqkgkrvb88la7lrrjogqlhkidr9m3hhet7f8p7zlnonpiczyeb3bkevi9kp4xvztsoxw5tsnl9sg+osqeosvlck0qpeo4estp5jw8ou/oi/pufexam87us0v+wpn6aywwobm=</latexit> <latexit sha1_base64="ogep1qiwgh7wefqptgar9csmc3a=">aaachnicbvdlsgmxfm3uv62vqks3wsk4kjnsx7gpunfzwdpcw4zm5rynzwsg5i5ysv/ejb/ixowk4er/xvsx0nydcsfn3ppknicrwqdrfjuzhcwl5zxsam5tfwnzk7+9c2fivhoo8ljguh4wa1ioqkjacfvea4sccbwgdzwq1+5bgxgrw+wn0ipyr4m24ayt5odp6n6jni9oe+ebb0yfw9gp7h//ejvqfsmvkoqe7kzw6oclbtedg84tb0okziqkn/9shjfpi2vmkhnt8nwew/zuffzcmndmdssm91ghgpyqfofpdcbzdembvulajrvdculy/e0ysmiyfhtyzohh18zwruj/tuak7bpwqkgkrvb88la7lrrjogqlhkidr9m3hhet7f8p7zlnonpiczyeb3bkevi9kp4xvztsoxw5tsnl9sg+osqeosvlck0qpeo4estp5jw8ou/oi/pufexam87us0v+wpn6aywwobm=</latexit> <latexit sha1_base64="ogep1qiwgh7wefqptgar9csmc3a=">aaachnicbvdlsgmxfm3uv62vqks3wsk4kjnsx7gpunfzwdpcw4zm5rynzwsg5i5ysv/ejb/ixowk4er/xvsx0nydcsfn3ppknicrwqdrfjuzhcwl5zxsam5tfwnzk7+9c2fivhoo8ljguh4wa1ioqkjacfvea4sccbwgdzwq1+5bgxgrw+wn0ipyr4m24ayt5odp6n6jni9oe+ebb0yfw9gp7h//ejvqfsmvkoqe7kzw6oclbtedg84tb0okziqkn/9shjfpi2vmkhnt8nwew/zuffzcmndmdssm91ghgpyqfofpdcbzdembvulajrvdculy/e0ysmiyfhtyzohh18zwruj/tuak7bpwqkgkrvb88la7lrrjogqlhkidr9m3hhet7f8p7zlnonpiczyeb3bkevi9kp4xvztsoxw5tsnl9sg+osqeosvlck0qpeo4estp5jw8ou/oi/pufexam87us0v+wpn6aywwobm=</latexit> <latexit sha1_base64="ogep1qiwgh7wefqptgar9csmc3a=">aaachnicbvdlsgmxfm3uv62vqks3wsk4kjnsx7gpunfzwdpcw4zm5rynzwsg5i5ysv/ejb/ixowk4er/xvsx0nydcsfn3ppknicrwqdrfjuzhcwl5zxsam5tfwnzk7+9c2fivhoo8ljguh4wa1ioqkjacfvea4sccbwgdzwq1+5bgxgrw+wn0ipyr4m24ayt5odp6n6jni9oe+ebb0yfw9gp7h//ejvqfsmvkoqe7kzw6oclbtedg84tb0okziqkn/9shjfpi2vmkhnt8nwew/zuffzcmndmdssm91ghgpyqfofpdcbzdembvulajrvdculy/e0ysmiyfhtyzohh18zwruj/tuak7bpwqkgkrvb88la7lrrjogqlhkidr9m3hhet7f8p7zlnonpiczyeb3bkevi9kp4xvztsoxw5tsnl9sg+osqeosvlck0qpeo4estp5jw8ou/oi/pufexam87us0v+wpn6aywwobm=</latexit> <latexit sha1_base64="pf/7dqf0u+btz4vaovag9vrfopc=">aaacgnicbvdlsgmxfm34rpvvdekmwarxzayikm6kblxwcgyhlummc9ugzjjdckcspf/hxl9x40lfnbjxb0wfi9p6iohk3hus3bomuhh03r9naxlldw09t5hf3nre2s3s7d+bjnmcfj7irnddzkakbt4klfbpnba4lfale9ejeu0btbgjusn+cq2ydzroc87qskghxa882ryktyrhhdavduenelceetvqosjiww4kh0gh6jbcmegi8aakskaobowvzptwllzmlpkxdc9nswxvrceldppnzedkei91oggpyjgy1ma825aewywi7utbpzco1vnhgmxg9opqdsymu2a+nhl/qzuybj+3bkklgylik4famasy0ffqnbiaomq+jyxryf9kezdpxthgmbchepmjlxk/xlooebenxcrvni0cosrh5ir45ixuya2pep9w8kreybt5d56dv+fd+zy0ljltzwh5a+f7f3hpobq=</latexit> <latexit sha1_base64="pf/7dqf0u+btz4vaovag9vrfopc=">aaacgnicbvdlsgmxfm34rpvvdekmwarxzayikm6kblxwcgyhlummc9ugzjjdckcspf/hxl9x40lfnbjxb0wfi9p6iohk3hus3bomuhh03r9naxlldw09t5hf3nre2s3s7d+bjnmcfj7irnddzkakbt4klfbpnba4lfale9ejeu0btbgjusn+cq2ydzroc87qskghxa882ryktyrhhdavduenelceetvqosjiww4kh0gh6jbcmegi8aakskaobowvzptwllzmlpkxdc9nswxvrceldppnzedkei91oggpyjgy1ma825aewywi7utbpzco1vnhgmxg9opqdsymu2a+nhl/qzuybj+3bkklgylik4famasy0ffqnbiaomq+jyxryf9kezdpxthgmbchepmjlxk/xlooebenxcrvni0cosrh5ir45ixuya2pep9w8kreybt5d56dv+fd+zy0ljltzwh5a+f7f3hpobq=</latexit> <latexit sha1_base64="pf/7dqf0u+btz4vaovag9vrfopc=">aaacgnicbvdlsgmxfm34rpvvdekmwarxzayikm6kblxwcgyhlummc9ugzjjdckcspf/hxl9x40lfnbjxb0wfi9p6iohk3hus3bomuhh03r9naxlldw09t5hf3nre2s3s7d+bjnmcfj7irnddzkakbt4klfbpnba4lfale9ejeu0btbgjusn+cq2ydzroc87qskghxa882ryktyrhhdavduenelceetvqosjiww4kh0gh6jbcmegi8aakskaobowvzptwllzmlpkxdc9nswxvrceldppnzedkei91oggpyjgy1ma825aewywi7utbpzco1vnhgmxg9opqdsymu2a+nhl/qzuybj+3bkklgylik4famasy0ffqnbiaomq+jyxryf9kezdpxthgmbchepmjlxk/xlooebenxcrvni0cosrh5ir45ixuya2pep9w8kreybt5d56dv+fd+zy0ljltzwh5a+f7f3hpobq=</latexit> <latexit sha1_base64="pf/7dqf0u+btz4vaovag9vrfopc=">aaacgnicbvdlsgmxfm34rpvvdekmwarxzayikm6kblxwcgyhlummc9ugzjjdckcspf/hxl9x40lfnbjxb0wfi9p6iohk3hus3bomuhh03r9naxlldw09t5hf3nre2s3s7d+bjnmcfj7irnddzkakbt4klfbpnba4lfale9ejeu0btbgjusn+cq2ydzroc87qskghxa882ryktyrhhdavduenelceetvqosjiww4kh0gh6jbcmegi8aakskaobowvzptwllzmlpkxdc9nswxvrceldppnzedkei91oggpyjgy1ma825aewywi7utbpzco1vnhgmxg9opqdsymu2a+nhl/qzuybj+3bkklgylik4famasy0ffqnbiaomq+jyxryf9kezdpxthgmbchepmjlxk/xlooebenxcrvni0cosrh5ir45ixuya2pep9w8kreybt5d56dv+fd+zy0ljltzwh5a+f7f3hpobq=</latexit> F X = F X1 F X2 F X3, F X4 F X5 Lets recall the rules on independence and are independent X 4 and X 5 are not independent X 5 X 4 115

<latexit sha1_base64="xbon6ondbb9h94lhsysynq7coza=">aaacnhicbva9t8mwehx4pnwvgfkskismkiligfgqlawmiffaqa0ix7lsc8eo7auiqvqnwpgftdawaglln+cudea5ydbzu3t3vhelulj0/wdvynjqemz2br60sli0vfjexbuyojmc6lxlbzorsycfgjoklnbmdbakktcibk7yfomwjbvaxwi/hu7crpxocs7quwh5rbkgth1i2wh3ogaqhuavzlj7qrqgsimvkoyu3kwqcq1ikxeamgpvahbccswv+qog4yaoqiuucr6wh9ux5lniwnljrg0ffoodnxeflzasttmlkem37bpadiqwgo0mrlsp6zzjytrvxp3rlxz7uzfgibx9jhkvccoe/zvlyf9yrqy7+52bugmgopj3og4mkwqaw0hjyycj7dvauhe2cmp7zdcozuismyh4u/i4qneqb9xgyrdydfy4muc2ycbzjghzi0fkljytouhknjyrv/lmpxgv3rv38v064rwadfirvm8v5n+p9w==</latexit> <latexit sha1_base64="xbon6ondbb9h94lhsysynq7coza=">aaacnhicbva9t8mwehx4pnwvgfkskismkiligfgqlawmiffaqa0ix7lsc8eo7auiqvqnwpgftdawaglln+cudea5ydbzu3t3vhelulj0/wdvynjqemz2br60sli0vfjexbuyojmc6lxlbzorsycfgjoklnbmdbakktcibk7yfomwjbvaxwi/hu7crpxocs7quwh5rbkgth1i2wh3ogaqhuavzlj7qrqgsimvkoyu3kwqcq1ikxeamgpvahbccswv+qog4yaoqiuucr6wh9ux5lniwnljrg0ffoodnxeflzasttmlkem37bpadiqwgo0mrlsp6zzjytrvxp3rlxz7uzfgibx9jhkvccoe/zvlyf9yrqy7+52bugmgopj3og4mkwqaw0hjyycj7dvauhe2cmp7zdcozuismyh4u/i4qneqb9xgyrdydfy4muc2ycbzjghzi0fkljytouhknjyrv/lmpxgv3rv38v064rwadfirvm8v5n+p9w==</latexit> <latexit sha1_base64="xbon6ondbb9h94lhsysynq7coza=">aaacnhicbva9t8mwehx4pnwvgfkskismkiligfgqlawmiffaqa0ix7lsc8eo7auiqvqnwpgftdawaglln+cudea5ydbzu3t3vhelulj0/wdvynjqemz2br60sli0vfjexbuyojmc6lxlbzorsycfgjoklnbmdbakktcibk7yfomwjbvaxwi/hu7crpxocs7quwh5rbkgth1i2wh3ogaqhuavzlj7qrqgsimvkoyu3kwqcq1ikxeamgpvahbccswv+qog4yaoqiuucr6wh9ux5lniwnljrg0ffoodnxeflzasttmlkem37bpadiqwgo0mrlsp6zzjytrvxp3rlxz7uzfgibx9jhkvccoe/zvlyf9yrqy7+52bugmgopj3og4mkwqaw0hjyycj7dvauhe2cmp7zdcozuismyh4u/i4qneqb9xgyrdydfy4muc2ycbzjghzi0fkljytouhknjyrv/lmpxgv3rv38v064rwadfirvm8v5n+p9w==</latexit> <latexit sha1_base64="xbon6ondbb9h94lhsysynq7coza=">aaacnhicbva9t8mwehx4pnwvgfkskismkiligfgqlawmiffaqa0ix7lsc8eo7auiqvqnwpgftdawaglln+cudea5ydbzu3t3vhelulj0/wdvynjqemz2br60sli0vfjexbuyojmc6lxlbzorsycfgjoklnbmdbakktcibk7yfomwjbvaxwi/hu7crpxocs7quwh5rbkgth1i2wh3ogaqhuavzlj7qrqgsimvkoyu3kwqcq1ikxeamgpvahbccswv+qog4yaoqiuucr6wh9ux5lniwnljrg0ffoodnxeflzasttmlkem37bpadiqwgo0mrlsp6zzjytrvxp3rlxz7uzfgibx9jhkvccoe/zvlyf9yrqy7+52bugmgopj3og4mkwqaw0hjyycj7dvauhe2cmp7zdcozuismyh4u/i4qneqb9xgyrdydfy4muc2ycbzjghzi0fkljytouhknjyrv/lmpxgv3rv38v064rwadfirvm8v5n+p9w==</latexit> <latexit sha1_base64="tc2t4mepq0e6w/vhwzpkfq+khmo=">aaacmhicbvbntwixeo36ififevtssew8kv3fqpfc9mirexesijtud4dgbrtpu0zc+ete/cfgcwc1xv0vdmepce7s5vxnm5noc2lothhdsbo0vlk6tp7byg9ube/sfvb2h7rmfiu6lvyqzka0ccagbpjh0iwvkcjg0ageb9n84wmuzllcm0emnyj0bosysoyl/ek16zdx+xq3dtybirhhkh01/fmzugfmioqy7cumplkelk2geeur6c/8qtetuzpai8dlqbflufmlb+1q0isylsknwrc8nzydo80wymgubycaykifsq9afgosge4mjxup8lflqtyvyp7jlyw7wzekkdadkldkiji+ns+l5h+5vmk6l50he3fiqndpog7csze4tq+htae1fgabocraqdhte0wossbnrqne/mqloh5auip5d+vi5szzi4co0re6qr66qbvurtvurxs9ohf0gt6dv2fsfdnfu+msk9ucod/h/pwcxq6oza==</latexit> <latexit sha1_base64="tc2t4mepq0e6w/vhwzpkfq+khmo=">aaacmhicbvbntwixeo36ififevtssew8kv3fqpfc9mirexesijtud4dgbrtpu0zc+ete/cfgcwc1xv0vdmepce7s5vxnm5noc2lothhdsbo0vlk6tp7byg9ube/sfvb2h7rmfiu6lvyqzka0ccagbpjh0iwvkcjg0ageb9n84wmuzllcm0emnyj0bosysoyl/ek16zdx+xq3dtybirhhkh01/fmzugfmioqy7cumplkelk2geeur6c/8qtetuzpai8dlqbflufmlb+1q0isylsknwrc8nzydo80wymgubycaykifsq9afgosge4mjxup8lflqtyvyp7jlyw7wzekkdadkldkiji+ns+l5h+5vmk6l50he3fiqndpog7csze4tq+htae1fgabocraqdhte0wossbnrqne/mqloh5auip5d+vi5szzi4co0re6qr66qbvurtvurxs9ohf0gt6dv2fsfdnfu+msk9ucod/h/pwcxq6oza==</latexit> <latexit sha1_base64="tc2t4mepq0e6w/vhwzpkfq+khmo=">aaacmhicbvbntwixeo36ififevtssew8kv3fqpfc9mirexesijtud4dgbrtpu0zc+ete/cfgcwc1xv0vdmepce7s5vxnm5noc2lothhdsbo0vlk6tp7byg9ube/sfvb2h7rmfiu6lvyqzka0ccagbpjh0iwvkcjg0ageb9n84wmuzllcm0emnyj0bosysoyl/ek16zdx+xq3dtybirhhkh01/fmzugfmioqy7cumplkelk2geeur6c/8qtetuzpai8dlqbflufmlb+1q0isylsknwrc8nzydo80wymgubycaykifsq9afgosge4mjxup8lflqtyvyp7jlyw7wzekkdadkldkiji+ns+l5h+5vmk6l50he3fiqndpog7csze4tq+htae1fgabocraqdhte0wossbnrqne/mqloh5auip5d+vi5szzi4co0re6qr66qbvurtvurxs9ohf0gt6dv2fsfdnfu+msk9ucod/h/pwcxq6oza==</latexit> <latexit sha1_base64="tc2t4mepq0e6w/vhwzpkfq+khmo=">aaacmhicbvbntwixeo36ififevtssew8kv3fqpfc9mirexesijtud4dgbrtpu0zc+ete/cfgcwc1xv0vdmepce7s5vxnm5noc2lothhdsbo0vlk6tp7byg9ube/sfvb2h7rmfiu6lvyqzka0ccagbpjh0iwvkcjg0ageb9n84wmuzllcm0emnyj0bosysoyl/ek16zdx+xq3dtybirhhkh01/fmzugfmioqy7cumplkelk2geeur6c/8qtetuzpai8dlqbflufmlb+1q0isylsknwrc8nzydo80wymgubycaykifsq9afgosge4mjxup8lflqtyvyp7jlyw7wzekkdadkldkiji+ns+l5h+5vmk6l50he3fiqndpog7csze4tq+htae1fgabocraqdhte0wossbnrqne/mqloh5auip5d+vi5szzi4co0re6qr66qbvurtvurxs9ohf0gt6dv2fsfdnfu+msk9ucod/h/pwcxq6oza==</latexit> F X = F X1 F X2 F X3, F X4 F X5 Lets recall the rules on independence and are not independent conditioned on X 4 and X 5 are independent conditioned on X 5 X 4 116

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback