Inference. Data. Model. Variates

Transcription

1

2 Data Inference Variates Model ˆθ (,..., )

3 mˆθn(d) m θ2 M m θ1

4 (,, ) (,,, ) (,, ) α

5

6

7 = :=: (, )

8 F( ) = = {(, ),, } F( ) X( ) = Γ( ) = Σ = ( ) = ( ) ( ) = { = } :=: (U, )

9 , = { = } = { = } x 2 e i, e j = δ j i e 2 e 1 e 2 e 1 x 1 =,, =, =,, =,, =, Σ =, Σ =

10 ,, (, ) R (, ) =, =, ( ) =,, = =, ( ) (U, ) (, ) = ( ) ( ) ( ) = ( ) ( ) ( ) =, = κ( ), Σ =

11 Γ = :=: (, ) Γ := ( ) F( ) ( )

12 ( ) ( ) ( ) = ( ) =, = ( ) ( ) ( ) v p c(t) p v q q v q = c v p M

13 γ γ = γ( ) γ γ = γ( ) = (γ ( )) γ ( ) + Γ γ ( ) γ ( ) =, γ ( ) = γ( ) Γ =

14 , =, +, { } =, +,,, ( ) =, ( ) ( ) ( ) ( ) ( ).

15 Γ Σ = ( + ) (, ) = ( (, )) + ( (, )) ( (, )) + ([, ], ) ([, ], ) ([, ], ).

16 (, ) = [, ] [, ]( ) = ( ( )) ( ( )) F( ) = Γ ( ) = Γ = Γ = [, ]

17

18

19 {, } {, } := C = C =, =, +,, ( ) = ( ) ( ), ( ) ( ) ( ).

20 (,,, ) {, } := + = = Γ Γ = σ( )σ( )σ( ) (,, ) =,, (,, ) =, { α } α R = = = = Γ α = Γ α, Γ α = Γ + α, Γ Σ = Γ (,, { α, α } ) α R

21 α κ = C κ α α α (,, α ) (,, α = ( α ) ) α = ± (,, ) (,, ) α (, ) α (, ) = α ( (, ) (, ) )

22 Conjugate Connection Manifolds (M, g,, ) (M, g, C = Γ Γ) (M, g, α, α ) (M, g, αc) I[p θ : p θ ] = D(θ : θ ) Parametric families f-divergences Expected Manifold (M, Fisher g, α, α ) α-geometry Divergence Manifold (M, D g, D, D = D ) D flat D flat KL on exponential families KL on mixture families Conformal divergences on deformed families Etc. Cubic skewness tensor Cijk = E[ il jl kl] αc = αfisher g Bregman divergence Distance = Non-metric divergence α = 1+α α 2 Γ ±α = Γ α 2 C canonical divergence Dually flat Manifolds (M, F, F ) (Hessian Manifolds) Dual Legendre potentials Bregman Pythagorean theorem Smooth Manifolds LC = + 2 Self-dual Manifold Spherical Manifold Euclidean Manifold Fisher-Riemannian Manifold Multinomial family Riemannian Manifolds (M, g) = (M, g, LC ) Location family g = Fisher g Fisher gij = E[ il jl] Location-scale family Hyperbolic Manifold Distance = Metric geodesic length Frank Nielsen (,, α, α ) (,, α ) (,, α, α ) (,, α )

23

24 (θ : θ ) := (θ) (θ ) (θ θ ) (θ ) (,, ) := (,, ) := = [ (θ : θ ] ) θ =θ = (θ) Γ := Γ (θ) = := = (θ) ( ) = (,, )

25 (η) := sup θ Θ {θ η (θ)} η = (θ) θ = (η) (θ) (η) = { } /{ } (θ : θ ) =, (θ : η ) = (θ) + (η ) θ η =, (η : θ) (θ : θ ) := (θ : θ) = (η : η ) (η) = (η), :=: Γ (η) =, η = (η)

26 θ [ ] η [ ] = γ (P, Q) F γ(q, R) γ(p, Q) F γ (Q, R) P P R Q D(P : R) = D(P : Q) + D(Q : R) BF (θ(p ) : θ(r)) = BF (θ(p ) : θ(q)) + BF (θ(q) : θ(r)) Q D (P : R) = D (P : Q) + D (Q : R) BF (η(p ) : η(r)) = BF (η(p ) : η(q)) + BF (η(q) : η(r)) R γ (, ) γ(, ) (η( ) η( )) θ( ) θ( ) = γ(, ) γ (, ) (θ( ) θ( )) η( ) η( ) =

27 θ η (θ( ) : θ( ) ) = arg min (θ( ) : θ( )) (θ( ) : θ( ) ) = ( θ( ) : θ( )) = arg min (θ( ) : θ( ))

28 α

29 P := { θ ( )} θ θ Θ (θ; ) (θ; ) := log (θ; ) θ = θ = ( ) :=: θ (θ; ) dim(θ) = P (θ) := θ [ ] θ [ˆθ ( )] P (θ) X Θ

30 { ( ) } E = θ ( ) = exp ( )θ (θ) + ( ) θ Θ = E (θ) = θ [ ( )] = (θ) = (η) { } M = θ ( ) = θ ( ) + ( ) θ Θ = { ( )} X ( ) µ( ) = ( ) µ( ) = [ ] ( ) ( ) M (θ) = θ θ ( ) = X ( ) ( ) µ( ) θ ( )

31 α α P (, ) := ( ) θ P θ (θ)( ) θ P := θ [( )( )], P := θ [( + )( )] (P, P, P, P ) = θ [ ] α {(P, P, P α, P +α )} α PΓ α := + α [( = θ + α ) ] ( ) P α + P α = P := ( P ) E M M Γ = MΓ = E Γ = E Γ =

32

33 (θā : θ Ā ) (θ : θ ) A A p 1 p 2 p 3 p 4 p 5 p 6 p 7 p 8 p coarse graining p 1 + p 2 p 3 + p 4 + p 5 p 6 p 7 + p 8 p A > (θ : θ ) := ( ) θ θ ( ), ( ) = θ ( ) = λ ( ) := ( ) + λ( ) λ = ( ) =

34 = ( ) ( ; θ) = ( ( ); θ) [ ( ; θ) : ( ; θ )] = = X Y ( ( ; θ ) ) ( ; θ) µ( ) ( ; θ) ( ( ; θ ) ) ( ; θ) µ( ) ( ; θ) = [ ( ; θ) : ( ; θ )] [ ( ; θ) : ( ; θ )] = [ ( ; θ ) : ( ; θ)] = [ ( ; θ) : ( ; θ )] ( ) ( ) :=

35 α α [ : ] := α ( ) α ( ) +α ( ) µ( ) ( ) = ( +α α ) [ : ] = ( ) log ( ) ( ) µ( ) ( ) = log α = [ : ] = ( ) log ( ) ( ) µ( ) ( ) = log α = [ : ] = ( ( ) ( )) µ( ) ( ) = ( ) α = ( ( ) log ( ) ( ) + ( ) log ( ) + ( ) ) µ( ) ( ) + ( ) ( ) = ( + ) log + + log ( ) ( ) µ( ) ( ) =

36 α [ ( ; θ) : ( ; θ + θ)] = ( ) ( ; θ + θ) ( ; θ) µ( ) ( ; θ) Σ = (θ) θ θ P (θ : θ ) := P [ ( ; θ) : ( ; θ )] P ( : ) P ( : ) = [ : ] ± P := P ( ) P := P ±α α = ( ) + + / /

37 (,,, ) (,, ) (,, α, α ) (,, α )

38

39

40 [ θ : θ ] = (θ : θ) : = = p 0 (x) p 1 (x) x 1 x 2 x θ, θ E E = { θ } (E, E, E, E ) (E, ) [ θ : θ ]

41 α [, ] = log min α (, ) X α ( ) α ( ) µ( ), E (α) (θ : θ ) = α (θ ) + ( α) (θ ) (θ + ( α)θ ), [ θ : θ ] = (θ : θ (α ) ) = (θ : θ (α ) )

42 = θ = (, ) (, ) m-bisector Bi m (P θ1, P θ2 ) η-coordinate system p θ 12 e-geodesic G e (P θ1, P θ2 ) P θ 12 p θ2 p θ1 C(θ 1 : θ 2 ) = B(θ 1 : θ 12)

43 E,..., E η-coordinate system Chernoff distribution between natural neighbours

44 M

45 + {,..., } M M = { θ ( ) = = ( )} M1 M2 Gaussian(-2,1) Cauchy(2,1) Laplace(0,1) (M, M, M, M ) ( θ, ) [ θ : θ ] = (θ : θ ), (θ) = ( θ ) = θ ( ) log θ ( ) µ( )

46 = ( ( ; θ)) X

47

48 Dissimilarity measure C 3 Divergence v-divergence D v Projective divergence double sided one-sided scaled conformal divergence CD,g( : ; ) γ-divergence D(λp : λ p ) = D(p : p ) Hÿvarinen SM/RM D(λp : p ) = D(p : p ) conformal divergence CD,g( : ) scaled Bregman divergence BF ( : ; ) total Bregman divergence tb( : ) Bregman divergence BF ( : ) Csiszár f-divergence If( : ) D v (P : Q) = D(v(P ) : v(q)) If (P : Q) = ( ) p(x)f ( q(x) p(x) dν(x) BF (P : Q) = F (P ) F (Q) P Q, F (Q) BF (P :Q) tbf (P : Q) = 1+ F (Q) 2 CD,g(P : Q) = g(q)d(p : Q) ( ) P BF,g(P : Q; W ) = W BF Q : Q W

49 (,,, ) α (,,, ) (,, )

50 α = + β β >