Normal Factor Graphs, Linear Algebra and Probabilistic Modeling
|
|
|
- Aron Allison
- 5 years ago
- Views:
Transcription
1 Normal Factor Graphs, Linear Algebra and Probabilistic Modeling Yongyi Mao University of Ottawa Workshop on Counting, Inference, and Optimization on Graphs Princeton University, NJ, USA November 3, 2011
2 It started on day one of my PhD program...
3 August 1, PhD advisor Frank Kschischang. Codes on graphs: normal realizations [Forney, 2001] Kschischang: Go read it! and went on a vacation...
4 August 1, PhD advisor Frank Kschischang. Codes on graphs: normal realizations [Forney, 2001] Kschischang: Go read it! and went on a vacation...
5 August 1, PhD advisor Frank Kschischang. Codes on graphs: normal realizations [Forney, 2001] Kschischang: Go read it! and went on a vacation...
6 August 1, PhD advisor Frank Kschischang. Codes on graphs: normal realizations [Forney, 2001] Kschischang: Go read it! and went on a vacation...
7 Normal graph duality x 1 C 1 ^x 1 C 2 C K 1 C K 2 x 2 C 3 C 4 ^x 2 C K 3 C K 4 C 5 C K 5 Duality theorem: Dual normal graphs represent dual codes
8 One month later... Kschischang: Why do normal graphs have duality, but factor graphs do not? Two weeks later... Me: We need... Fourier... we need convolution..." Given f(x, y) and g(y, z) f(x, y) g(y, z) = w f(x, y w)g(w, z)
9 One month later... Kschischang: Why do normal graphs have duality, but factor graphs do not? Two weeks later... Me: We need... Fourier... we need convolution..." Given f(x, y) and g(y, z) f(x, y) g(y, z) = w f(x, y w)g(w, z)
10 One month later... Kschischang: Why do normal graphs have duality, but factor graphs do not? Two weeks later... Me: We need... Fourier... we need convolution..." Given f(x, y) and g(y, z) f(x, y) g(y, z) = w f(x, y w)g(w, z)
11 Convolutional factor graph bipartite graph: function nodes and variables nodes encodes a convolutional product f 1 x 1 f 2 x 2 f 3 x 3 f 1 (x 1, x 2 ) f 2 (x 1, x 3 ) f 3 (x 1, x 3 )
12 Factor graph duality Dual FGs: (G, {f i }, ) and (G, {^f i }, ) F Dual FGs encode a FT pair Dual FGs of codes: (G, {δ Ci }, ) and (G, {δ C K i }, ) δ C F δc K Dual FGs represent dual codes Normal graph duality reinterpreted On factor graphs and the Fourier transform [Mao & Kschischang, ISIT2001, IT2005]
13 Factor graph duality Dual FGs: (G, {f i }, ) and (G, {^f i }, ) F Dual FGs encode a FT pair Dual FGs of codes: (G, {δ Ci }, ) and (G, {δ C K i }, ) δ C F δc K Dual FGs represent dual codes Normal graph duality reinterpreted On factor graphs and the Fourier transform [Mao & Kschischang, ISIT2001, IT2005]
14 Factor graph duality Dual FGs: (G, {f i }, ) and (G, {^f i }, ) F Dual FGs encode a FT pair Dual FGs of codes: (G, {δ Ci }, ) and (G, {δ C K i }, ) δ C F δc K Dual FGs represent dual codes Normal graph duality reinterpreted On factor graphs and the Fourier transform [Mao & Kschischang, ISIT2001, IT2005]
15 Factor graph duality Dual FGs: (G, {f i }, ) and (G, {^f i }, ) F Dual FGs encode a FT pair Dual FGs of codes: (G, {δ Ci }, ) and (G, {δ C K i }, ) δ C F δc K Dual FGs represent dual codes Normal graph duality reinterpreted On factor graphs and the Fourier transform [Mao & Kschischang, ISIT2001, IT2005]
16 Factor graph duality Dual FGs: (G, {f i }, ) and (G, {^f i }, ) F Dual FGs encode a FT pair Dual FGs of codes: (G, {δ Ci }, ) and (G, {δ C K i }, ) δ C F δc K Dual FGs represent dual codes Normal graph duality reinterpreted On factor graphs and the Fourier transform [Mao & Kschischang, ISIT2001, IT2005]
17 Factor graph duality Dual FGs: (G, {f i }, ) and (G, {^f i }, ) F Dual FGs encode a FT pair Dual FGs of codes: (G, {δ Ci }, ) and (G, {δ C K i }, ) δ C F δc K Dual FGs represent dual codes Normal graph duality reinterpreted On factor graphs and the Fourier transform [Mao & Kschischang, ISIT2001, IT2005]
18 Factor graph duality Dual FGs: (G, {f i }, ) and (G, {^f i }, ) F Dual FGs encode a FT pair Dual FGs of codes: (G, {δ Ci }, ) and (G, {δ C K i }, ) δ C F δc K Dual FGs represent dual codes Normal graph duality reinterpreted On factor graphs and the Fourier transform [Mao & Kschischang, ISIT2001, IT2005]
19 The reviewer... Dave Forney 9 pages of review Forney: It may take years for people to take it [convolutional FG] and run
20 The reviewer... Dave Forney 9 pages of review Forney: It may take years for people to take it [convolutional FG] and run
21 The reviewer... Dave Forney 9 pages of review Forney: It may take years for people to take it [convolutional FG] and run
22 Going probabilistic... Bredan Frey Convolutional FGs as probabilistic models [Mao, Kschischang & Frey, UAI 2004] Each node: joint distribution of a collection of (latent) variables All collections are independent Convoltional CFG represents the joint distribution of observed variables that are linearly transformed from the latent variables
23 Going probabilistic... Bredan Frey Convolutional FGs as probabilistic models [Mao, Kschischang & Frey, UAI 2004] Each node: joint distribution of a collection of (latent) variables All collections are independent Convoltional CFG represents the joint distribution of observed variables that are linearly transformed from the latent variables
24 Going probabilistic... Bredan Frey Convolutional FGs as probabilistic models [Mao, Kschischang & Frey, UAI 2004] Each node: joint distribution of a collection of (latent) variables All collections are independent Convoltional CFG represents the joint distribution of observed variables that are linearly transformed from the latent variables
25 Going probabilistic... Bredan Frey Convolutional FGs as probabilistic models [Mao, Kschischang & Frey, UAI 2004] Each node: joint distribution of a collection of (latent) variables All collections are independent Convoltional CFG represents the joint distribution of observed variables that are linearly transformed from the latent variables
26 Linear characteristic graphical models December 2010, from Danny Bickson: Inference with multivariate heavy-tails in linear models [Bickson & Guestrin, NIPS2010] modelling heavy-tail (stable) distributions applications and inference algorithms based on convolutional FG and duality
27 Linear characteristic graphical models December 2010, from Danny Bickson: Inference with multivariate heavy-tails in linear models [Bickson & Guestrin, NIPS2010] modelling heavy-tail (stable) distributions applications and inference algorithms based on convolutional FG and duality
28 Linear characteristic graphical models December 2010, from Danny Bickson: Inference with multivariate heavy-tails in linear models [Bickson & Guestrin, NIPS2010] modelling heavy-tail (stable) distributions applications and inference algorithms based on convolutional FG and duality
29 Linear characteristic graphical models December 2010, from Danny Bickson: Inference with multivariate heavy-tails in linear models [Bickson & Guestrin, NIPS2010] modelling heavy-tail (stable) distributions applications and inference algorithms based on convolutional FG and duality
30 Missed opportunity... Year met Frey in Toronto Frey: How about a model for CDF? Cumulative distribution networks and derivativesum-product algorithm [Huang & Frey, UAI2008] F X1X 2 x 1 F X1X 3 x 2 F X1X 3 x 3 CDN: multiplicative FG each function node: a CDF the global function is a CDF efficiently solves structured ranking problems
31 Missed opportunity... Year met Frey in Toronto Frey: How about a model for CDF? Cumulative distribution networks and derivativesum-product algorithm [Huang & Frey, UAI2008] F X1X 2 x 1 F X1X 3 x 2 F X1X 3 x 3 CDN: multiplicative FG each function node: a CDF the global function is a CDF efficiently solves structured ranking problems
32 Missed opportunity... Year met Frey in Toronto Frey: How about a model for CDF? Cumulative distribution networks and derivativesum-product algorithm [Huang & Frey, UAI2008] F X1X 2 x 1 F X1X 3 x 2 F X1X 3 x 3 CDN: multiplicative FG each function node: a CDF the global function is a CDF efficiently solves structured ranking problems
33 Missed opportunity... Year met Frey in Toronto Frey: How about a model for CDF? Cumulative distribution networks and derivativesum-product algorithm [Huang & Frey, UAI2008] F X1X 2 x 1 F X1X 3 x 2 F X1X 3 x 3 CDN: multiplicative FG each function node: a CDF the global function is a CDF efficiently solves structured ranking problems
34 Missed opportunity... Year met Frey in Toronto Frey: How about a model for CDF? Cumulative distribution networks and derivativesum-product algorithm [Huang & Frey, UAI2008] F X1X 2 x 1 F X1X 3 x 2 F X1X 3 x 3 CDN: multiplicative FG each function node: a CDF the global function is a CDF efficiently solves structured ranking problems
35 Conditional/marginal independence properties f 1 x 1 f 2 x 2 f 3 x 3 Multiplicative FG: conditional indepdence; Convolutional FG: marginal independence; CDN: marginal independence
36 Again, it is Frank Kschischang... Early Kschischang: "Look at this!" Constrained coding as networks of relations [Schwartz & Bruck 2008] Holographic algorithms [Valiant 2004] Counting problem ± ± Holographic reduction: Holant theorem Apparent normal structure Do something with holographic algorithm? My question: the magic of Holant theorem...
37 Again, it is Frank Kschischang... Early Kschischang: "Look at this!" Constrained coding as networks of relations [Schwartz & Bruck 2008] Holographic algorithms [Valiant 2004] Counting problem ± ± Holographic reduction: Holant theorem Apparent normal structure Do something with holographic algorithm? My question: the magic of Holant theorem...
38 Again, it is Frank Kschischang... Early Kschischang: "Look at this!" Constrained coding as networks of relations [Schwartz & Bruck 2008] Holographic algorithms [Valiant 2004] Counting problem ± ± Holographic reduction: Holant theorem Apparent normal structure Do something with holographic algorithm? My question: the magic of Holant theorem...
39 Again, it is Frank Kschischang... Early Kschischang: "Look at this!" Constrained coding as networks of relations [Schwartz & Bruck 2008] Holographic algorithms [Valiant 2004] Counting problem ± ± Holographic reduction: Holant theorem Apparent normal structure Do something with holographic algorithm? My question: the magic of Holant theorem...
40 Again, it is Frank Kschischang... Early Kschischang: "Look at this!" Constrained coding as networks of relations [Schwartz & Bruck 2008] Holographic algorithms [Valiant 2004] Counting problem ± ± Holographic reduction: Holant theorem Apparent normal structure Do something with holographic algorithm? My question: the magic of Holant theorem...
41 Give it the student... Ali Al-Bashebsheh Reformulated normal factor graphs Generalized Holant theorem Unified normal graph duality theorem and Holant theorem
42 Give it the student... Ali Al-Bashebsheh Reformulated normal factor graphs Generalized Holant theorem Unified normal graph duality theorem and Holant theorem
43 Two concurrent submissions... Editor Pascal Vontobel Normal factor graphs and holographic transformations " [Al-Bashebsheh & Mao 2011] Codes on graphs: duality and MacWilliams identities [Forney 2011] Extensive comments, rounds of rewriting
44 Two concurrent submissions... Editor Pascal Vontobel Normal factor graphs and holographic transformations " [Al-Bashebsheh & Mao 2011] Codes on graphs: duality and MacWilliams identities [Forney 2011] Extensive comments, rounds of rewriting
45 Normal factor graphs under new semantics... x 1 f x 3 1 f 2 x 4 x 5 x6 Z G (x 1, x 2 ) := x 2 f 3 f 4 x7 x 3,...,x 7 f 1 (x 1, x 3, x 4 )f 2 (x 3, x 5, x 6 )f 3 (x 2, x 4, x 5, x 7 )f 4 (x 6, x 7 ). := xf 1, f 2, f 3, f 4 y. := xf 1 (x 1, x 3, x 4 ), f 2 (x 3, x 5, x 6 ), f 3 (x 2, x 4, x 5, x 7 ), f 4 (x 6, x 7 ).y
46 Functions as multidimensional arrays X 1 = 2, X 2 = 3, and X 3 = 5. f x 1 x 2 x 3 f x 1 x 2 x 3 f x 1 x 2 x 3
47 Functions as multidimensional arrays X 1 = 2, X 2 = 3, and X 3 = 5. f x 1 x 2 x 3 f x 1 x 2 x 3 f x 1 x 2 x 3
48 Functions as multidimensional arrays X 1 = 2, X 2 = 3, and X 3 = 5. f x 1 x 2 x 3 f x 1 x 2 x 3 f x 1 x 2 x 3
49 Example (Equality Indicator) { 1, s = s δ(s, s 1 1 ) := 0, otherwise δ(, ): identity matrix Example ( Transformer ) Function Φ : χ χ C is called a transformer if it corresponds to an invertible matrix. Dual pair of transformers Φ and p Φ: an inverse pair of matrices xφ(s, t), p Φ(t, s 1 )y = δ(s, s 1 )
50 Example (Equality Indicator) { 1, s = s δ(s, s 1 1 ) := 0, otherwise δ(, ): identity matrix Example ( Transformer ) Function Φ : χ χ C is called a transformer if it corresponds to an invertible matrix. Dual pair of transformers Φ and p Φ: an inverse pair of matrices xφ(s, t), p Φ(t, s 1 )y = δ(s, s 1 )
51 Example (Equality Indicator) { 1, s = s δ(s, s 1 1 ) := 0, otherwise δ(, ): identity matrix Example ( Transformer ) Function Φ : χ χ C is called a transformer if it corresponds to an invertible matrix. Dual pair of transformers Φ and p Φ: an inverse pair of matrices xφ(s, t), p Φ(t, s 1 )y = δ(s, s 1 )
52 Example (Equality Indicator) { 1, s = s δ(s, s 1 1 ) := 0, otherwise δ(, ): identity matrix Example ( Transformer ) Function Φ : χ χ C is called a transformer if it corresponds to an invertible matrix. Dual pair of transformers Φ and p Φ: an inverse pair of matrices xφ(s, t), p Φ(t, s 1 )y = δ(s, s 1 )
53 Example (Equality Indicator) { 1, s = s δ(s, s 1 1 ) := 0, otherwise δ(, ): identity matrix Example ( Transformer ) Function Φ : χ χ C is called a transformer if it corresponds to an invertible matrix. Dual pair of transformers Φ and p Φ: an inverse pair of matrices xφ(s, t), p Φ(t, s 1 )y = δ(s, s 1 )
54 Functions as linear maps xf, gy: xf(s), g(s)y: vector-vector dot product xf(s), g(s, t)y: vector-matrix product xf(s, t), g(t, u)y: matrix-matrix product xf(s), g(t)y: vector outer product, matrix Kronecker product, tensor product etc NFGs are linear algebraic expressions written graphically.
55 Functions as linear maps xf, gy: xf(s), g(s)y: vector-vector dot product xf(s), g(s, t)y: vector-matrix product xf(s, t), g(t, u)y: matrix-matrix product xf(s), g(t)y: vector outer product, matrix Kronecker product, tensor product etc NFGs are linear algebraic expressions written graphically.
56 Functions as linear maps xf, gy: xf(s), g(s)y: vector-vector dot product xf(s), g(s, t)y: vector-matrix product xf(s, t), g(t, u)y: matrix-matrix product xf(s), g(t)y: vector outer product, matrix Kronecker product, tensor product etc NFGs are linear algebraic expressions written graphically.
57 Functions as linear maps xf, gy: xf(s), g(s)y: vector-vector dot product xf(s), g(s, t)y: vector-matrix product xf(s, t), g(t, u)y: matrix-matrix product xf(s), g(t)y: vector outer product, matrix Kronecker product, tensor product etc NFGs are linear algebraic expressions written graphically.
58 Functions as linear maps xf, gy: xf(s), g(s)y: vector-vector dot product xf(s), g(s, t)y: vector-matrix product xf(s, t), g(t, u)y: matrix-matrix product xf(s), g(t)y: vector outer product, matrix Kronecker product, tensor product etc NFGs are linear algebraic expressions written graphically.
59 Opening/closing the box [Vontobel, Loeliger, 2002] f xf, gy g X X X Y X Opening/closing the box is a powerful technique
60 Opening/closing the box [Vontobel, Loeliger, 2002] f xf, gy g X X X Y X Opening/closing the box is a powerful technique
61 Holographic transformation x 1 y 1 f 1 f 2 f 1 f 2 x 2 f 3 f 4 y 2 f 3 f 4 f 5 f 5 Z G Z G H Theorem (Generalized Holant Theorem) Z G H is externally transformed version of Z G.
62 Generalized Holant Theorem When external transformers are δ(, ) reduces to Valiant s Holant Theorem When all transformers are Fourier kernels reduces to General NFG Duality Theorem when each function is a subgroup-indicator function reduces to Forney s Normal Graph Duality Theorem
63 Generalized Holant Theorem When external transformers are δ(, ) reduces to Valiant s Holant Theorem When all transformers are Fourier kernels reduces to General NFG Duality Theorem when each function is a subgroup-indicator function reduces to Forney s Normal Graph Duality Theorem
64 Generalized Holant Theorem When external transformers are δ(, ) reduces to Valiant s Holant Theorem When all transformers are Fourier kernels reduces to General NFG Duality Theorem when each function is a subgroup-indicator function reduces to Forney s Normal Graph Duality Theorem
65 General NFG Duality Theorem x 1 ^x 1 f 1 f 2 F[f1] F[f2] x 2 f 3 f 4 ^x 2 F[f3] F[f4] f 5 F[f5] x 1 f 1 f 2 ^x 1 f 1 f 2 x 2 f 3 f 4 ^x 2 f 3 f 4 f 5 f 5
66 Holographic algorithm: PerfMatch(H) H: weighted graph PerfMatch(H) π(h) := ¹ MPQ(H) epm w(e), where Q(H) the set of all perfect matchings of H. If H is planar, π(h) is poly. solvable by FKT algorithm Matchgate (G, U), where G is a weighted graph, U V(G). Signature of matchgate (G, U) is a function µ(x U )... a? 1 2 b? 3 c a 1{2 1 b 2 3 c a a? 1? 1{ b c b c 2
67 Holographic algorithm: PerfMatch(H) H: weighted graph PerfMatch(H) π(h) := ¹ MPQ(H) epm w(e), where Q(H) the set of all perfect matchings of H. If H is planar, π(h) is poly. solvable by FKT algorithm Matchgate (G, U), where G is a weighted graph, U V(G). Signature of matchgate (G, U) is a function µ(x U )... a? 1 2 b? 3 c a 1{2 1 b 2 3 c a a? 1? 1{ b c b c 2
68 Holographic algorithm: PerfMatch(H) H: weighted graph PerfMatch(H) π(h) := ¹ MPQ(H) epm w(e), where Q(H) the set of all perfect matchings of H. If H is planar, π(h) is poly. solvable by FKT algorithm Matchgate (G, U), where G is a weighted graph, U V(G). Signature of matchgate (G, U) is a function µ(x U )... a? 1 2 b? 3 c a 1{2 1 b 2 3 c a a? 1? 1{ b c b c 2
69 Holographic algorithm: a graph-theoretic property Lemma: π(h) = x E ± m i=1 µ i(x E(i) ). 1 a a? 1? 1{ b c b c 2 1 a? 1 2 b? 3 c a 1{2 1 3 b c a a? 1? 1{ b c b c 2 1 a? 1 2 b? 3 c a 1{2 1 3 b c 2
70 Holographic algorithm Solve Z := x E ±v f v(x E(v) ) a? 1 2 b? 3 c a 1{2 1 3 b c 2 a? 1 2 b? 3 c a 1{2 1 3 b c 2 FKT
71 Don t forget the mathematicians... Vontobel pointed to Trace Diagram, Birdtracks... Unshackling linear algebra from linear notation [Peterson 2009] Group Theory: Birdtracks, Lie s, and Exceptional Group, [Cvitanovic, 2008] Trace diagram is contained in NFG framework Trace diagrams are NFGs with degree 1, 2, or χ NFGs: a diagrammatic approach to linear algebra [Al-Bashebsheh, Mao & Vontobel ISIT2011]
72 Don t forget the mathematicians... Vontobel pointed to Trace Diagram, Birdtracks... Unshackling linear algebra from linear notation [Peterson 2009] Group Theory: Birdtracks, Lie s, and Exceptional Group, [Cvitanovic, 2008] Trace diagram is contained in NFG framework Trace diagrams are NFGs with degree 1, 2, or χ NFGs: a diagrammatic approach to linear algebra [Al-Bashebsheh, Mao & Vontobel ISIT2011]
73 Don t forget the mathematicians... Vontobel pointed to Trace Diagram, Birdtracks... Unshackling linear algebra from linear notation [Peterson 2009] Group Theory: Birdtracks, Lie s, and Exceptional Group, [Cvitanovic, 2008] Trace diagram is contained in NFG framework Trace diagrams are NFGs with degree 1, 2, or χ NFGs: a diagrammatic approach to linear algebra [Al-Bashebsheh, Mao & Vontobel ISIT2011]
74 Don t forget the mathematicians... Vontobel pointed to Trace Diagram, Birdtracks... Unshackling linear algebra from linear notation [Peterson 2009] Group Theory: Birdtracks, Lie s, and Exceptional Group, [Cvitanovic, 2008] Trace diagram is contained in NFG framework Trace diagrams are NFGs with degree 1, 2, or χ NFGs: a diagrammatic approach to linear algebra [Al-Bashebsheh, Mao & Vontobel ISIT2011]
75 Ciliated function nodes Draw the edges of f counter-clockwise, with first edge ciliated x n x n 1 x k+1... f... x n x n 1 x k+1... f... x 1 x 2 x k x 1 x 2 x k f(x 1,..., x n ) f(x k+1,..., x n, x 1,..., x k )
76 Ciliated function nodes x 1 x 1 x 1 x 1 A A A A B B B B x 2 x 2 x 2 x 2 A B A B T A T B T A T B
77 Trace A i Z = i A(i, i) = tr(a)
78 tr(a B) = tr(b A) Proof: A B Read the graph in two ways: Z = tr(a B) and Z = tr(b A). l
79 Levi-Civita symbol S n : the permutation group on {1,..., n}, n = χ. Levi-Civita symbol, ε : {1,..., n} n C s.t. ( ) ( ) 1 n 1 n sgn, ε(x 1,..., x n )= x 1 x n x 1 x n 0, otherwise Contraction (n = 3). t P S n ε(y 1, y 2, t)ε(t, x 2, x 1 ) = δ(x 1, y 2 )δ(x 2, y 1 ) δ(x 1, y 1 )δ(x 2, y 2 ). x 1 x 2 ε t ε = x 1 x 2 y 1 y 2 x 1 x 2 y 1 y 2 y 1 y 2
80 Levi-Civita symbol S n : the permutation group on {1,..., n}, n = χ. Levi-Civita symbol, ε : {1,..., n} n C s.t. ( ) ( ) 1 n 1 n sgn, ε(x 1,..., x n )= x 1 x n x 1 x n 0, otherwise Contraction (n = 3). t P S n ε(y 1, y 2, t)ε(t, x 2, x 1 ) = δ(x 1, y 2 )δ(x 2, y 1 ) δ(x 1, y 1 )δ(x 2, y 2 ). x 1 x 2 ε t ε = x 1 x 2 y 1 y 2 x 1 x 2 y 1 y 2 y 1 y 2
81 Levi-Civita symbol S n : the permutation group on {1,..., n}, n = χ. Levi-Civita symbol, ε : {1,..., n} n C s.t. ( ) ( ) 1 n 1 n sgn, ε(x 1,..., x n )= x 1 x n x 1 x n 0, otherwise Contraction (n = 3). t P S n ε(y 1, y 2, t)ε(t, x 2, x 1 ) = δ(x 1, y 2 )δ(x 2, y 1 ) δ(x 1, y 1 )δ(x 2, y 2 ). x 1 x 2 ε t ε = x 1 x 2 y 1 y 2 x 1 x 2 y 1 y 2 y 1 y 2
82 Vector cross product u, v : {1, 2, 3} F u, v P F 3. Cross Product. Z(x) = t 1,t 2 u(t 1 )v(t 2 )ε(t 1, t 2, x). Z(1) = u(2)v(3) u(3)v(2) Z(2) = u(3)v(1) u(1)v(3) Z(3) = u(1)v(2) u(2)v(1) x ε t 1 t 2 u v Z = u v.
83 Vector cross product u, v : {1, 2, 3} F u, v P F 3. Cross Product. Z(x) = t 1,t 2 u(t 1 )v(t 2 )ε(t 1, t 2, x). Z(1) = u(2)v(3) u(3)v(2) Z(2) = u(3)v(1) u(1)v(3) Z(3) = u(1)v(2) u(2)v(1) x ε t 1 t 2 u v Z = u v.
84 Vector cross product u, v : {1, 2, 3} F u, v P F 3. Cross Product. Z(x) = t 1,t 2 u(t 1 )v(t 2 )ε(t 1, t 2, x). Z(1) = u(2)v(3) u(3)v(2) Z(2) = u(3)v(1) u(1)v(3) Z(3) = u(1)v(2) u(2)v(1) x ε t 1 t 2 u v Z = u v.
85 Vector cross product u, v : {1, 2, 3} F u, v P F 3. Cross Product. Z(x) = t 1,t 2 u(t 1 )v(t 2 )ε(t 1, t 2, x). Z(1) = u(2)v(3) u(3)v(2) Z(2) = u(3)v(1) u(1)v(3) Z(3) = u(1)v(2) u(2)v(1) x ε t 1 t 2 u v Z = u v.
86 Vector cross product u, v : {1, 2, 3} F u, v P F 3. Cross Product. Z(x) = t 1,t 2 u(t 1 )v(t 2 )ε(t 1, t 2, x). Z(1) = u(2)v(3) u(3)v(2) Z(2) = u(3)v(1) u(1)v(3) Z(3) = u(1)v(2) u(2)v(1) x ε t 1 t 2 u v Z = u v.
87 Cross product identities (u v) (s w) = ((u v) s) w = (w (u v)) s = ((s w) u) v = (v (s w)) u = (u s)(v w) (u w)(v s). Proof: w s ε ε u v = w s ε ε u v = w s ε ε u v = w s ε ε u v = w s ε ε u v = w s ε ε u v = w s u v w s u v
88 Determinant ± n A := sgn(σ) σps n j=1 A(j, σ(j)). Lemma: ε = A ε A... A...
89 A B = A B Proof: AB ε... = AB ε... AB = B A ε A... B = A B ε... B = A B ε...
90 A T = A Proof: ε ε ε ε A T... =... A T A T =... A A = A... ε ε ε ε
91 Pfaffian A is a 2n 2n skew-symmetric matrix. ± Pf(A) = 1 2 n n! sgn(σ) n A ( σ(2i 1), σ(2i) ). σps 2n i=1 ε A A... A Z = n! 2 n Pf(A) Affirms a conjecture of Peterson [Peterson, 2009].
92 Pfaffian and Determinant ( ) 0 In J :=, A := R T J R for some R. Then Pf(A) 2 = A. I n 0 Proof: Pf(J) = 1 Pf(A) ε J J... J = ε A A... A R R R = ε J J... J R R R R R R = ε J J... J R R R = R ε J J... J Pf(A) = R Pf(A) 2 = R 2 = A.
93 Going probabilistic... NFGs as probabilistic model [Al-Bashebsheh & Mao, 2011, soon available on arxiv] Internal edges: latent variabels External edges: observed variables Exterior function: joint distribution of observed variables (up to scale) Function: compatibility/constraints/interaction protential source of randomness, i.e., distribution Bipartite normal: WLOG
94 Going probabilistic... NFGs as probabilistic model [Al-Bashebsheh & Mao, 2011, soon available on arxiv] Internal edges: latent variabels External edges: observed variables Exterior function: joint distribution of observed variables (up to scale) Function: compatibility/constraints/interaction protential source of randomness, i.e., distribution Bipartite normal: WLOG
95 Going probabilistic... NFGs as probabilistic model [Al-Bashebsheh & Mao, 2011, soon available on arxiv] Internal edges: latent variabels External edges: observed variables Exterior function: joint distribution of observed variables (up to scale) Function: compatibility/constraints/interaction protential source of randomness, i.e., distribution Bipartite normal: WLOG
96 Special functions x 1 x 1 x 1 Σ max x 2 x n x 2 x n x 2 x n
97 Cumulus and difference transformers A D χ: an ordered set (integers) with well defined. A(x, x 1 ) = [x 1 x] [ ] 1 0 E.g. χ = 2, A = ; χ = 3, A = xf X (x 1 ), A(x, x 1 )y = F X (x) D := A 1, e.g.d = for χ =
98 Bipartite NFG reduces to FG One partitiion: arbitrary potential functions The other partition: equality indicators NFG model reduces to FG model f1 x 1 f 1 x 1 f2 x 2 f 2 x 2 f3 x 3 f 3 x 3
99 Bipartite NFG reduces to convolutional FG One side: distributions of indep. collections of latent RVs The other side: sum indicators No RVs from the same collection connect to the same indicator NFG model reduces to convolutional FG model f 1 x 1 f 1 x 1 f2 x 2 f 2 x 2 x f 3 3 f 3 x 3 f(x, u)g(v, y)[w = u + v] = f(x, u)g(w u, y) = f(x, w) g(w, y) u,v u
100 CDN is a holographic transformed NFG Lemma x 1 A max x 2... xn = A x 1 = x 2... A x n Corollary x 1 A D max x 2 D x n = x 1 x 2 x n
101 CDN is a holographic transformed NFG One side: distributions of indep. collections of latent RVs The other side: max indicators No RVs from the same collection connect to the same indicator f 1 max x 1 f 2 x 2 f 3 max x 3
102 CDN is a holographic transformed NFG x 2 A f 1 f 2 f 3 max max x 1 x 2 x 3 f1 f 2 f3 A A A A A D D D D D max max A A x 1 x 3 F X1 X 2 x 1 F X1 X 2 x 1 F X1 X 3 x 2 F X1 X 3 x 2 F X1 X 3 x 3 F X1 X 3 x 3
103 The independence coincidence of Convolutional FG and CDN f 1 max x 1 f 1 x 1 f 2 x 2 f2 x 2 f 3 max x 3 f 3 x 3 Both are generative models In fact, the independence property holds when changing indicator to conditional distributions Changing the right-side functions gives rise to a family of infinite models... what applications?
104 The End NFGs are linear algebraic expressions written graphically Opening/closing the box is a powerful technique NFGs are potential tools for inference Outlook: continuous alphabets
105 The End NFGs are linear algebraic expressions written graphically Opening/closing the box is a powerful technique NFGs are potential tools for inference Outlook: continuous alphabets
106 The End NFGs are linear algebraic expressions written graphically Opening/closing the box is a powerful technique NFGs are potential tools for inference Outlook: continuous alphabets
107 The End NFGs are linear algebraic expressions written graphically Opening/closing the box is a powerful technique NFGs are potential tools for inference Outlook: continuous alphabets
108 The End NFGs are linear algebraic expressions written graphically Opening/closing the box is a powerful technique NFGs are potential tools for inference Outlook: continuous alphabets
109
Codes on Graphs, Normal Realizations, and Partition Functions
Codes on Graphs, Normal Realizations, and Partition Functions G. David Forney, Jr. 1 Workshop on Counting, Inference and Optimization Princeton, NJ November 3, 2011 1 Joint work with: Heide Gluesing-Luerssen,
Partition Functions of Normal Factor Graphs
Partition Functions of Normal Factor Graphs G. David Forney, r. Laboratory for nformation and Decision Systems Massachusetts nstitute of Technology Cambridge, M 02139, US forneyd@comcast.net Pascal O.
Some Results on Matchgates and Holographic Algorithms. Jin-Yi Cai Vinay Choudhary University of Wisconsin, Madison. NSF CCR and CCR
Some Results on Matchgates and Holographic Algorithms Jin-Yi Cai Vinay Choudhary University of Wisconsin, Madison NSF CCR-0208013 and CCR-0511679. 1 #P Counting problems: #SAT: How many satisfying assignments
A Complexity Classification of the Six Vertex Model as Holant Problems
A Complexity Classification of the Six Vertex Model as Holant Problems Jin-Yi Cai (University of Wisconsin-Madison) Joint work with: Zhiguo Fu, Shuai Shao and Mingji Xia 1 / 84 Three Frameworks for Counting
Basis Collapse in Holographic Algorithms Over All Domain Sizes
Basis Collapse in Holographic Algorithms Over All Domain Sizes Sitan Chen Harvard College March 29, 2016 Introduction Matchgates/Holographic Algorithms: A Crash Course Basis Size and Domain Size Collapse
Codes on graphs. Chapter Elementary realizations of linear block codes
Chapter 11 Codes on graphs In this chapter we will introduce the subject of codes on graphs. This subject forms an intellectual foundation for all known classes of capacity-approaching codes, including
15-251: Great Theoretical Ideas In Computer Science Recitation 9 : Randomized Algorithms and Communication Complexity Solutions
15-251: Great Theoretical Ideas In Computer Science Recitation 9 : Randomized Algorithms and Communication Complexity Solutions Definitions We say a (deterministic) protocol P computes f if (x, y) {0,
On Symmetric Signatures in Holographic Algorithms. Jin-Yi Cai University of Wisconsin, Madison. Pinyan Lu Tsinghua University, Beijing
On Symmetric Signatures in Holographic Algorithms Jin-Yi Cai University of Wisconsin, Madison Pinyan Lu Tsinghua University, Beijing NSF CCR-0208013 and CCR-0511679. 1 #P Counting problems: #SAT: How many
Advanced Combinatorial Optimization Feb 13 and 25, Lectures 4 and 6
18.438 Advanced Combinatorial Optimization Feb 13 and 25, 2014 Lectures 4 and 6 Lecturer: Michel X. Goemans Scribe: Zhenyu Liao and Michel X. Goemans Today, we will use an algebraic approach to solve the
arxiv:cs/ v2 [cs.it] 1 Oct 2006
![arxiv:cs/ v2 [cs.it] 1 Oct 2006](/thumbs/89/97933712.jpg "arxiv:cs/ v2 [cs.it] 1 Oct 2006")
A General Computation Rule for Lossy Summaries/Messages with Examples from Equalization Junli Hu, Hans-Andrea Loeliger, Justin Dauwels, and Frank Kschischang arxiv:cs/060707v [cs.it] 1 Oct 006 Abstract
26 : Spectral GMs. Lecturer: Eric P. Xing Scribes: Guillermo A Cidre, Abelino Jimenez G.
10-708: Probabilistic Graphical Models, Spring 2015 26 : Spectral GMs Lecturer: Eric P. Xing Scribes: Guillermo A Cidre, Abelino Jimenez G. 1 Introduction A common task in machine learning is to work with
Formalizing Randomized Matching Algorithms
Formalizing Randomized Matching Algorithms Stephen Cook Joint work with Dai Tri Man Lê Department of Computer Science University of Toronto Canada The Banff Workshop on Proof Complexity 2011 1 / 15 Feasible
Properties of Continuous Probability Distributions The graph of a continuous probability distribution is a curve. Probability is represented by area
Properties of Continuous Probability Distributions The graph of a continuous probability distribution is a curve. Probability is represented by area under the curve. The curve is called the probability
REPRESENTATION THEORY OF S n
REPRESENTATION THEORY OF S n EVAN JENKINS Abstract. These are notes from three lectures given in MATH 26700, Introduction to Representation Theory of Finite Groups, at the University of Chicago in November
Complexity Dichotomies for Counting Problems
Complexity Dichotomies for Counting Problems Jin-Yi Cai Computer Sciences Department University of Wisconsin Madison, WI 53706 Email: jyc@cs.wisc.edu Xi Chen Department of Computer Science Columbia University
Chordal structure in computer algebra: Permanents
Chordal structure in computer algebra: Permanents Diego Cifuentes Laboratory for Information and Decision Systems Electrical Engineering and Computer Science Massachusetts Institute of Technology Joint
Extending Monte Carlo Methods to Factor Graphs with Negative and Complex Factors
ITW 202 Extending Monte Carlo Methods to Factor Graphs with Negative and Complex Factors Mehdi Molkaraie and Hans-Andrea Loeliger ETH Zurich Dept. of Information Technology & Electrical Engineering 8092
Siegel s theorem, edge coloring, and a holant dichotomy
Siegel s theorem, edge coloring, and a holant dichotomy Tyson Williams (University of Wisconsin-Madison) Joint with: Jin-Yi Cai and Heng Guo (University of Wisconsin-Madison) Appeared at FOCS 2014 1 /
Marginal Densities, Factor Graph Duality, and High-Temperature Series Expansions
Marginal Densities, Factor Graph Duality, and High-Temperature Series Expansions Mehdi Molkaraie mehdi.molkaraie@alumni.ethz.ch arxiv:90.02733v [stat.ml] 7 Jan 209 Abstract We prove that the marginals
Introduction Eigen Values and Eigen Vectors An Application Matrix Calculus Optimal Portfolio. Portfolios. Christopher Ting.
Portfolios Christopher Ting Christopher Ting http://www.mysmu.edu/faculty/christophert/ : christopherting@smu.edu.sg : 6828 0364 : LKCSB 5036 November 4, 2016 Christopher Ting QF 101 Week 12 November 4,
An Importance Sampling Algorithm for Models with Weak Couplings
An Importance ampling Algorithm for Models with Weak Couplings Mehdi Molkaraie EH Zurich mehdi.molkaraie@alumni.ethz.ch arxiv:1607.00866v1 [cs.i] 4 Jul 2016 Abstract We propose an importance sampling algorithm
Minimal-span bases, linear system theory, and the invariant factor theorem
Minimal-span bases, linear system theory, and the invariant factor theorem G. David Forney, Jr. MIT Cambridge MA 02139 USA DIMACS Workshop on Algebraic Coding Theory and Information Theory DIMACS Center,
LIMITATION OF LEARNING RANKINGS FROM PARTIAL INFORMATION. By Srikanth Jagabathula Devavrat Shah
00 AIM Workshop on Ranking LIMITATION OF LEARNING RANKINGS FROM PARTIAL INFORMATION By Srikanth Jagabathula Devavrat Shah Interest is in recovering distribution over the space of permutations over n elements
Dichotomy Theorems in Counting Complexity and Holographic Algorithms. Jin-Yi Cai University of Wisconsin, Madison. Supported by NSF CCF
Dichotomy Theorems in Counting Complexity and Holographic Algorithms Jin-Yi Cai University of Wisconsin, Madison September 7th, 204 Supported by NSF CCF-27549. The P vs. NP Question It is generally conjectured
Math 121 Homework 4: Notes on Selected Problems
Math 121 Homework 4: Notes on Selected Problems 11.2.9. If W is a subspace of the vector space V stable under the linear transformation (i.e., (W ) W ), show that induces linear transformations W on W
Lecture 5 January 16, 2013
UBC CPSC 536N: Sparse Approximations Winter 2013 Prof. Nick Harvey Lecture 5 January 16, 2013 Scribe: Samira Samadi 1 Combinatorial IPs 1.1 Mathematical programs { min c Linear Program (LP): T x s.t. a
Linear Algebra Homework and Study Guide
Linear Algebra Homework and Study Guide Phil R. Smith, Ph.D. February 28, 20 Homework Problem Sets Organized by Learning Outcomes Test I: Systems of Linear Equations; Matrices Lesson. Give examples of
Computational Complexity Theory. The World of P and NP. Jin-Yi Cai Computer Sciences Dept University of Wisconsin, Madison
Computational Complexity Theory The World of P and NP Jin-Yi Cai Computer Sciences Dept University of Wisconsin, Madison Sept 11, 2012 Supported by NSF CCF-0914969. 1 2 Entscheidungsproblem The rigorous
A Framework for the Construction of Golay Sequences
1 A Framework for the Construction of Golay Sequences Frank Fiedler, Jonathan Jedwab, and Matthew G Parker Abstract In 1999 Davis and Jedwab gave an explicit algebraic normal form for m! h(m+) ordered
Problems in Linear Algebra and Representation Theory
Problems in Linear Algebra and Representation Theory (Most of these were provided by Victor Ginzburg) The problems appearing below have varying level of difficulty. They are not listed in any specific
Constructing Linkages for Drawing Plane Curves
Constructing Linkages for Drawing Plane Curves Christoph Koutschan (joint work with Matteo Gallet, Zijia Li, Georg Regensburger, Josef Schicho, Nelly Villamizar) Johann Radon Institute for Computational
The Complexity of Planar Boolean #CSP with Complex Weights
The Complexity of Planar Boolean #CSP with Complex Weights Heng Guo University of Wisconsin-Madison hguo@cs.wisc.edu Tyson Williams University of Wisconsin-Madison tdw@cs.wisc.edu Abstract We prove a complexity
Oeding (Auburn) tensors of rank 5 December 15, / 24
Oeding (Auburn) 2 2 2 2 2 tensors of rank 5 December 15, 2015 1 / 24 Recall Peter Burgisser s overview lecture (Jan Draisma s SIAM News article). Big Goal: Bound the computational complexity of det n,
Recoverabilty Conditions for Rankings Under Partial Information
Recoverabilty Conditions for Rankings Under Partial Information Srikanth Jagabathula Devavrat Shah Abstract We consider the problem of exact recovery of a function, defined on the space of permutations
DOMINO TILING. Contents 1. Introduction 1 2. Rectangular Grids 2 Acknowledgments 10 References 10
DOMINO TILING KASPER BORYS Abstract In this paper we explore the problem of domino tiling: tessellating a region with x2 rectangular dominoes First we address the question of existence for domino tilings
Lecture 3: Vectors. In Song Kim. September 1, 2011
Lecture 3: Vectors In Song Kim September 1, 211 1 Solving Equations Up until this point we have been looking at different types of functions, often times graphing them. Each point on a graph is a solution
Factor Graphs and Message Passing Algorithms Part 1: Introduction
Factor Graphs and Message Passing Algorithms Part 1: Introduction Hans-Andrea Loeliger December 2007 1 The Two Basic Problems 1. Marginalization: Compute f k (x k ) f(x 1,..., x n ) x 1,..., x n except
Math 110, Spring 2015: Midterm Solutions
Math 11, Spring 215: Midterm Solutions These are not intended as model answers ; in many cases far more explanation is provided than would be necessary to receive full credit. The goal here is to make
Expectation Propagation in Factor Graphs: A Tutorial
DRAFT: Version 0.1, 28 October 2005. Do not distribute. Expectation Propagation in Factor Graphs: A Tutorial Charles Sutton October 28, 2005 Abstract Expectation propagation is an important variational
Picard s Existence and Uniqueness Theorem
Picard s Existence and Uniqueness Theorem Denise Gutermuth 1 These notes on the proof of Picard s Theorem follow the text Fundamentals of Differential Equations and Boundary Value Problems, 3rd edition,
PROBLEMS ON LINEAR ALGEBRA
1 Basic Linear Algebra PROBLEMS ON LINEAR ALGEBRA 1. Let M n be the (2n + 1) (2n + 1) for which 0, i = j (M n ) ij = 1, i j 1,..., n (mod 2n + 1) 1, i j n + 1,..., 2n (mod 2n + 1). Find the rank of M n.
Constrained Codes as Networks of Relations
Constrained Codes as Networks of Relations Moshe Schwartz, Member, IEEE, and Jehoshua Bruck, Fellow, IEEE Abstract We address the well-known problem of determining the capacity of constrained coding systems.
22 : Hilbert Space Embeddings of Distributions
10-708: Probabilistic Graphical Models 10-708, Spring 2014 22 : Hilbert Space Embeddings of Distributions Lecturer: Eric P. Xing Scribes: Sujay Kumar Jauhar and Zhiguang Huo 1 Introduction and Motivation
On Weight Enumerators and MacWilliams Identity for Convolutional Codes
On Weight Enumerators and MacWilliams Identity for Convolutional Codes Irina E Bocharova 1, Florian Hug, Rolf Johannesson, and Boris D Kudryashov 1 1 Dept of Information Systems St Petersburg Univ of Information
McGill University Department of Mathematics and Statistics. Ph.D. preliminary examination, PART A. PURE AND APPLIED MATHEMATICS Paper BETA
McGill University Department of Mathematics and Statistics Ph.D. preliminary examination, PART A PURE AND APPLIED MATHEMATICS Paper BETA 17 August, 2018 1:00 p.m. - 5:00 p.m. INSTRUCTIONS: (i) This paper
Introduction to the Tensor Train Decomposition and Its Applications in Machine Learning
Introduction to the Tensor Train Decomposition and Its Applications in Machine Learning Anton Rodomanov Higher School of Economics, Russia Bayesian methods research group (http://bayesgroup.ru) 14 March
MATH 2331 Linear Algebra. Section 1.1 Systems of Linear Equations. Finding the solution to a set of two equations in two variables: Example 1: Solve:
MATH 2331 Linear Algebra Section 1.1 Systems of Linear Equations Finding the solution to a set of two equations in two variables: Example 1: Solve: x x = 3 1 2 2x + 4x = 12 1 2 Geometric meaning: Do these
CSci 8980: Advanced Topics in Graphical Models Gaussian Processes
CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee November 15, 2007 Gaussian Processes Outline Gaussian Processes Outline Parametric Bayesian Regression Gaussian
Correlation: Copulas and Conditioning
Correlation: Copulas and Conditioning This note reviews two methods of simulating correlated variates: copula methods and conditional distributions, and the relationships between them. Particular emphasis
Multivariate Normal-Laplace Distribution and Processes
CHAPTER 4 Multivariate Normal-Laplace Distribution and Processes The normal-laplace distribution, which results from the convolution of independent normal and Laplace random variables is introduced by
Mean-field dual of cooperative reproduction
The mean-field dual of systems with cooperative reproduction joint with Tibor Mach (Prague) A. Sturm (Göttingen) Friday, July 6th, 2018 Poisson construction of Markov processes Let (X t ) t 0 be a continuous-time
CS168: The Modern Algorithmic Toolbox Lecture #10: Tensors, and Low-Rank Tensor Recovery
CS168: The Modern Algorithmic Toolbox Lecture #10: Tensors, and Low-Rank Tensor Recovery Tim Roughgarden & Gregory Valiant May 3, 2017 Last lecture discussed singular value decomposition (SVD), and we
Introduction to Quantitative Techniques for MSc Programmes SCHOOL OF ECONOMICS, MATHEMATICS AND STATISTICS MALET STREET LONDON WC1E 7HX
Introduction to Quantitative Techniques for MSc Programmes SCHOOL OF ECONOMICS, MATHEMATICS AND STATISTICS MALET STREET LONDON WC1E 7HX September 2007 MSc Sep Intro QT 1 Who are these course for? The September
The Maximum Likelihood Threshold of a Graph
The Maximum Likelihood Threshold of a Graph Elizabeth Gross and Seth Sullivant San Jose State University, North Carolina State University August 28, 2014 Seth Sullivant (NCSU) Maximum Likelihood Threshold
Algorithms, Probability, and Computing Final Exam HS14
Institute of Theoretical Computer Science Thomas Holenstein, Emo Welzl, Peter Widmayer Algorithms, Probability, and Computing Final Exam HS14 Candidate First name:.........................................................
On Symmetric Signatures in Holographic Algorithms
On Symmetric Signatures in Holographic Algorithms Jin-Yi Cai a and Pinyan Lu b a Computer Sciences Department, University of Wisconsin Madison, WI 53706, USA jyc@cs.wisc.edu b Department of Computer Science
MILNOR SEMINAR: DIFFERENTIAL FORMS AND CHERN CLASSES
MILNOR SEMINAR: DIFFERENTIAL FORMS AND CHERN CLASSES NILAY KUMAR In these lectures I want to introduce the Chern-Weil approach to characteristic classes on manifolds, and in particular, the Chern classes.
Linear Response for Approximate Inference
Linear Response for Approximate Inference Max Welling Department of Computer Science University of Toronto Toronto M5S 3G4 Canada welling@cs.utoronto.ca Yee Whye Teh Computer Science Division University
Generators of certain inner mapping groups
Department of Algebra Charles University in Prague 3rd Mile High Conference on Nonassociative Mathematics, August 2013 Inner Mapping Group Definitions In a loop Q, the left and right translations by an
Vector spaces, duals and endomorphisms
Vector spaces, duals and endomorphisms A real vector space V is a set equipped with an additive operation which is commutative and associative, has a zero element 0 and has an additive inverse v for any
Learning from Sensor Data: Set II. Behnaam Aazhang J.S. Abercombie Professor Electrical and Computer Engineering Rice University
Learning from Sensor Data: Set II Behnaam Aazhang J.S. Abercombie Professor Electrical and Computer Engineering Rice University 1 6. Data Representation The approach for learning from data Probabilistic
Inference with Multivariate Heavy-Tails in Linear Models
Inference with Multivariate Heavy-Tails in Linear Models Danny Bickson and Carlos Guestrin Machine Learning Department Carnegie Mellon University Pittsburgh, PA 1513 bickson,guestrin}@cs.cmu.edu Abstract
Review Packet 1 B 11 B 12 B 13 B = B 21 B 22 B 23 B 31 B 32 B 33 B 41 B 42 B 43
Review Packet. For each of the following, write the vector or matrix that is specified: a. e 3 R 4 b. D = diag{, 3, } c. e R 3 d. I. For each of the following matrices and vectors, give their dimension.
CO 250 Final Exam Guide
Spring 2017 CO 250 Final Exam Guide TABLE OF CONTENTS richardwu.ca CO 250 Final Exam Guide Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4,
Chap. 1. Some Differential Geometric Tools
Chap. 1. Some Differential Geometric Tools 1. Manifold, Diffeomorphism 1.1. The Implicit Function Theorem ϕ : U R n R n p (0 p < n), of class C k (k 1) x 0 U such that ϕ(x 0 ) = 0 rank Dϕ(x) = n p x U
Convex and Semidefinite Programming for Approximation
Convex and Semidefinite Programming for Approximation We have seen linear programming based methods to solve NP-hard problems. One perspective on this is that linear programming is a meta-method since
Math 671: Tensor Train decomposition methods
Math 671: Eduardo Corona 1 1 University of Michigan at Ann Arbor December 8, 2016 Table of Contents 1 Preliminaries and goal 2 Unfolding matrices for tensorized arrays The Tensor Train decomposition 3
Reverse Edge Cut-Set Bounds for Secure Network Coding
Reverse Edge Cut-Set Bounds for Secure Network Coding Wentao Huang and Tracey Ho California Institute of Technology Michael Langberg University at Buffalo, SUNY Joerg Kliewer New Jersey Institute of Technology
Chapter 1. Matrix Algebra
ST4233, Linear Models, Semester 1 2008-2009 Chapter 1. Matrix Algebra 1 Matrix and vector notation Definition 1.1 A matrix is a rectangular or square array of numbers of variables. We use uppercase boldface
The Classification Program II: Tractable Classes and Hardness Proof
The Classification Program II: Tractable Classes and Hardness Proof Jin-Yi Cai (University of Wisconsin-Madison) January 27, 2016 1 / 104 Kasteleyn s Algorithm and Matchgates Previously... By a Pfaffian
Tail-Biting Trellis Realizations and Local Reductions
Noname manuscript No. (will be inserted by the editor) Tail-Biting Trellis Realizations and Local Reductions Heide Gluesing-Luerssen Received: date / Accepted: date Abstract This paper investigates tail-biting
1.1 GRAPHS AND LINEAR FUNCTIONS
MATHEMATICS EXTENSION 4 UNIT MATHEMATICS TOPIC 1: GRAPHS 1.1 GRAPHS AND LINEAR FUNCTIONS FUNCTIONS The concept of a function is already familiar to you. Since this concept is fundamental to mathematics,
THE EULER CHARACTERISTIC OF A LIE GROUP
THE EULER CHARACTERISTIC OF A LIE GROUP JAY TAYLOR 1 Examples of Lie Groups The following is adapted from [2] We begin with the basic definition and some core examples Definition A Lie group is a smooth
Previously Monte Carlo Integration
Previously Simulation, sampling Monte Carlo Simulations Inverse cdf method Rejection sampling Today: sampling cont., Bayesian inference via sampling Eigenvalues and Eigenvectors Markov processes, PageRank
Support Vector Machines.
Support Vector Machines www.cs.wisc.edu/~dpage 1 Goals for the lecture you should understand the following concepts the margin slack variables the linear support vector machine nonlinear SVMs the kernel
1 Matrices and Systems of Linear Equations. a 1n a 2n
March 31, 2013 16-1 16. Systems of Linear Equations 1 Matrices and Systems of Linear Equations An m n matrix is an array A = (a ij ) of the form a 11 a 21 a m1 a 1n a 2n... a mn where each a ij is a real
Summary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016
8. For any two events E and F, P (E) = P (E F ) + P (E F c ). Summary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016 Sample space. A sample space consists of a underlying
Higher Secants of Sato s Grassmannian. Jan Draisma TU Eindhoven
Higher Secants of Sato s Grassmannian Jan Draisma TU Eindhoven SIAM AG, Fort Collins, August 2, 2013 Grassmannians: functoriality and duality V finite-dimensional vector space Gr p (V) := {v 1 v p v i
ABSTRACT ALGEBRA WITH APPLICATIONS
ABSTRACT ALGEBRA WITH APPLICATIONS IN TWO VOLUMES VOLUME I VECTOR SPACES AND GROUPS KARLHEINZ SPINDLER Darmstadt, Germany Marcel Dekker, Inc. New York Basel Hong Kong Contents f Volume I Preface v VECTOR
Graphs & Algorithms: Advanced Topics Nowhere-Zero Flows
Graphs & Algorithms: Advanced Topics Nowhere-Zero Flows Uli Wagner ETH Zürich Flows Definition Let G = (V, E) be a multigraph (allow loops and parallel edges). An (integer-valued) flow on G (also called
Dichotomy for Holant Problems of Boolean Domain
Dichotomy for Holant Problems of Boolean Domain Jin-Yi Cai Pinyan Lu Mingji Xia Abstract Holant problems are a general framework to study counting problems. Both counting Constraint Satisfaction Problems
The prototypes of smooth manifolds
The prototypes of smooth manifolds The prototype smooth manifolds are the open subsets of R n. If U is an open subset of R n, a smooth map from U to R m is an m-tuple of real valued functions (f 1, f 2,...,
7 Matrix Operations. 7.0 Matrix Multiplication + 3 = 3 = 4
7 Matrix Operations Copyright 017, Gregory G. Smith 9 October 017 The product of two matrices is a sophisticated operations with a wide range of applications. In this chapter, we defined this binary operation,
LINES IN P 3. Date: December 12,
LINES IN P 3 Points in P 3 correspond to (projective equivalence classes) of nonzero vectors in R 4. That is, the point in P 3 with homogeneous coordinates [X : Y : Z : W ] is the line [v] spanned by the
Statistical Pattern Recognition
Statistical Pattern Recognition A Brief Mathematical Review Hamid R. Rabiee Jafar Muhammadi, Ali Jalali, Alireza Ghasemi Spring 2012 http://ce.sharif.edu/courses/90-91/2/ce725-1/ Agenda Probability theory
Transformation of Probability Densities
Transformation of Probability Densities This Wikibook shows how to transform the probability density of a continuous random variable in both the one-dimensional and multidimensional case. In other words,
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions
Algebra Exam Topics. Updated August 2017
Algebra Exam Topics Updated August 2017 Starting Fall 2017, the Masters Algebra Exam will have 14 questions. Of these students will answer the first 8 questions from Topics 1, 2, and 3. They then have
Math Camp II. Calculus. Yiqing Xu. August 27, 2014 MIT
Math Camp II Calculus Yiqing Xu MIT August 27, 2014 1 Sequence and Limit 2 Derivatives 3 OLS Asymptotics 4 Integrals Sequence Definition A sequence {y n } = {y 1, y 2, y 3,..., y n } is an ordered set
Math 52H: Multilinear algebra, differential forms and Stokes theorem. Yakov Eliashberg
Math 52H: Multilinear algebra, differential forms and Stokes theorem Yakov Eliashberg March 202 2 Contents I Multilinear Algebra 7 Linear and multilinear functions 9. Dual space.........................................
2 Functions of random variables
2 Functions of random variables A basic statistical model for sample data is a collection of random variables X 1,..., X n. The data are summarised in terms of certain sample statistics, calculated as
Statistics for scientists and engineers
Statistics for scientists and engineers February 0, 006 Contents Introduction. Motivation - why study statistics?................................... Examples..................................................3
PHYS 705: Classical Mechanics. Rigid Body Motion Introduction + Math Review
1 PHYS 705: Classical Mechanics Rigid Body Motion Introduction + Math Review 2 How to describe a rigid body? Rigid Body - a system of point particles fixed in space i r ij j subject to a holonomic constraint:
CPSC 540: Machine Learning
CPSC 540: Machine Learning Multivariate Gaussians Mark Schmidt University of British Columbia Winter 2019 Last Time: Multivariate Gaussian http://personal.kenyon.edu/hartlaub/mellonproject/bivariate2.html
3 Applications of partial differentiation
Advanced Calculus Chapter 3 Applications of partial differentiation 37 3 Applications of partial differentiation 3.1 Stationary points Higher derivatives Let U R 2 and f : U R. The partial derivatives
Commutative Banach algebras 79
8. Commutative Banach algebras In this chapter, we analyze commutative Banach algebras in greater detail. So we always assume that xy = yx for all x, y A here. Definition 8.1. Let A be a (commutative)
Consistent Bivariate Distribution
A Characterization of the Normal Conditional Distributions MATSUNO 79 Therefore, the function ( ) = G( : a/(1 b2)) = N(0, a/(1 b2)) is a solu- tion for the integral equation (10). The constant times of
Here we used the multiindex notation:
Mathematics Department Stanford University Math 51H Distributions Distributions first arose in solving partial differential equations by duality arguments; a later related benefit was that one could always
Symmetric 0-1 matrices with inverses having two distinct values and constant diagonal
Symmetric 0-1 matrices with inverses having two distinct values and constant diagonal Wayne Barrett Department of Mathematics, Brigham Young University, Provo, UT, 84602, USA Steve Butler Dept. of Mathematics,
Linear Codes, Target Function Classes, and Network Computing Capacity
Linear Codes, Target Function Classes, and Network Computing Capacity Rathinakumar Appuswamy, Massimo Franceschetti, Nikhil Karamchandani, and Kenneth Zeger IEEE Transactions on Information Theory Submitted: