Estimators, escort probabilities, and φ-exponential families in statistical physics
|
|
- Kenneth Peters
- 6 years ago
- Views:
Transcription
1 Estimators, escort probabilities, and φ-exponential families in statistical physics arxiv:math-ph/425v 4 Feb 24 Jan Naudts Departement Natuurkunde, Universiteit Antwerpen UIA, Universiteitsplein, 26 Antwerpen, Belgium Jan.Naudts@ua.ac.be Abstract The lower bound of Cramer and Rao is generalized to pairs of families of probability distributions, one of which is escort to the other. This bound is optimal for certain families, called φ-exponential in the paper. Their dual structure is explored. Introduction Aim of this paper is to translate some new results of statistical physics into the language of statistics. It is well-known that the exponential family of probability distribution functions (pdfs) plays a central role in statistical physics. When Gibbs introduced the canonical ensemble in 9 [] he postulated a distribution of energies E of the form p(e) = exp(g βe) () where G is a normalization constant and where the control parameter β is the inverse temperature. Only recently [2], a proposal was made to replace () by a more general family of pdfs. The resulting domain of research is known under the name of Tsallis thermostatistics. Some of the pdfs of Tsallis thermostatistics are known in statistics under the name of Amari s α-family [3]. The latter have been introduced in the context of geometry of statistical manifolds [4]. The appearance of the same family of pdfs in both domains is not accidental. The apparent link between both domains is clarified in the present paper.
2 The new notion introduced in Tsallis thermostatistics is that of pairs of families of pdfs, one of which is escort of the other [5]. Some basic concepts of statistics can be generalized by replacing at well-chosen places the pdf by its escort. In particular, we show in the next section how to generalize Fisher s information and, correspondingly, how to generalize the well-known lower bound of Cramer and Rao. Section 3 studies the statistical manifold of a family for which there exists an escort family satisfying the condition under which the generalized Cramer-Rao bound is optimal. This optimizing family has an affine geometry. Since this is usually the characteristic property of an exponential family a generalization of the latter seems indicated. Section 4 shows how a strictly positive non-decreasing function φ of R + determines a function which shares some properties with the natural logarithm and therefore is called below a φ-logarithm. The inverse function is called the φ-exponential. In Section 5 it is used to define the φ-exponential family in the obvious way, by replacing the exponential function exp by the φ-exponential function. The standard exponential family is then recovered by the choice φ(x) = x, the α-family of Amari by φ(x) = x (+α)/2, the equilibrium pdfs of Tsallis thermostatistics by the choice φ(x) = x q. The next three sections are used to establish the dual parametrization of the φ-exponential family and to discover the role of entropy functionals. Section 6 introduces a divergence of the Bregman type. In Section 7 it is used to prove the existence of an information function (or entropy functional) which is maximized by the φ-exponential pdfs. Section 8 introduces dual parameters in statistical physics these are energy and temperature. The paper ends with a short discussion in Section 9. There have been already some attempts to study Tsallis thermostatistics from a geometrical point of view. Trasarti-Battistoni [6] conjectured a deep connection between non-extensivity and geometry. He also gives general references to the use of geometric ideas in statistical physics. Several authors [7, 8, 9] have introduced a divergence belonging to Csiszár s class of f-divergences, which leads to a generalization of the Fisher information metric adapted to the context of Tsallis thermostatistics. The relation with the present work is unclear since here the geometry is determined by a divergence of the Bregman type. Also the recent work of Abe [] seems to be unrelated. 2
3 2 Estimators and escort pdfs Fix a measure space, µ. Let M (µ) denote the convex set of all probability distribution functions (pdfs) p normalized w.r.t. µ dµ(x) p(x) =. (2) Expectations w.r.t. p are denoted E p E p f = dµ(x) p(x)f(x). (3) Fix an open domain D of R n. Consider a family of pdfs p θ, parametrized with θ in D. The notation E θ will be used instead of E pθ. Simultaneously, a second family of pdfs (P θ ) θ D is considered. It is called the escort family. The notation F θ will be used instead of E Pθ. Recall that the Fisher information is given by ( ) ( ) I kl (θ) = E θ θ log(p θ) k θ log(p θ) l p θ p θ = dµ(x) p θ (x) θ k θ. (4) l A generalization, involving the two families of pdfs, is p θ p θ g kl (θ) = dµ(x) P θ (x) θ k θ. (5) l Clearly, the expression coincides with (4) if P θ = p θ. The following definition is a slight generalization of the usual definition of an unbiased estimator. Definition An estimator of the family (p θ ) θ D is a vector of random variables c k with the property that there exists a function F such that E θ c k = F(θ), k =,, n. (6) θk The function F will be called the scale function of the estimator. The estimator is unbiased if F(θ) = 2 θ kθ k so that E θ c k = θ k. The well-known lower bound of Cramer and Rao can be written as u k u l [E θ c k c l (E θ c k )(E θ c l )] [ uk v l 2 F θ k θ l ] 2 v k v l I kl (θ), (7) for arbitrary u and v in R n. A similar lower bound, involving the information matrix g kl instead of Fisher s I kl, is now formulated. 3
4 Theorem Let be given two families of pdfs (p θ ) θ D and (P θ ) θ D and corresponding expectations E θ and F θ. Let c be an estimator of (p θ ) θ D, with scale function F. Assume the regularity condition F θ P θ (x) θ p θ(x) = (8) k holds. Let g kl (θ) be the information matrix introduced before. Then, for all u and v in R n is u k u l [F θ c k c l (F θ c k )(F θ c l )] [ uk v l 2 θ l θ k F(θ) ] 2 v k v l g kl (θ). (9) The bound is optimal (in the sense that equality holds whenever u = v) if there exist a normalization function Z > and a function G such that θ kp θ(x) = Z(θ)P θ (x) θ k [G(θ) θl c l (x)] () holds for all k in [..m], for all θ D, and for µ-almost all x. In that case, c is an estimator of (P θ ) θ D with scale function G F θ c k = G θk. () Proof Let X k = P θ θ k p θ and Y k = c k F θ c k. (2) From Schwartz s inequality follows ( Fθ u k Y k v l X l ) 2 ( F θ u k Y k u l Y l )( Fθ v k X k v l X l ). (3) The l.h.s. equals, using (8), ( Fθ u k Y k v l X l ) 2 = = ( u k v l θ E θc l k (u k v l 2 ) 2 θ l θ kf(θ) ) 2. (4) The first factor of the r.h.s. equals F θ u k Y k u l Y l = u k u l [F θ c k c l (F θ c k )(F θ c l )]. (5) 4
5 The second factor of the r.h.s. equals F θ v k X k v l X l = v k v l g kl (θ). (6) This proves (9). Assume now that () holds. Combining it with the regularity condition (8) shows that c is an estimator for the escort family, with scaling function G. This makes it possible to write () as In this way one obtains On the other hand is 2 θ l θ k F(θ) = Z(θ)P θ (x) θ kp θ(x) = F θ c k c k (x). (7) u k u l [F θ c k c l (F θ c k )(F θ c l )] = uk u l g kl (θ) Z(θ) 2. (8) θ E θc l k = dµ(x) p θ θ (x)c k(x) l = Z(θ) dµ(x) P θ (x)c k (x) θ [G(θ) k θl c l (x)] = Z(θ) [F θ c k c l (F θ c k )(F θ c l )]. (9) Together with (8) this shows equality in (9) whenever u = v. It is not investigated whether () is a necessary condition. For practical application of the lower bound one has to assume that c is also an estimator of the escort family (P θ ) θ D, with scale function G. The previous proposition shows that this is automatically the case when () is satisfied. Example Let µ be the Lebesgue measure restricted to [, + ) and let p θ (x) = 2 [ x ] (2) θ θ + with θ > and [u] + = max{u, }. The Fisher information I(θ) is divergent. Hence, the usual lower bound of Cramer and Rao is useless. Consider now the escort family P θ (x) = θ e x/θ. (2) 5
6 Then one calculates g(θ) = 4 θ2(5e 3). (22) This fixes the r.h.s. of the inequality (9). Let us estimate θ via its first moment, with c(x) = 3x. One has E θ c = θ, E θ c 2 = (3/2)θ 2, F(θ) = θ 2 /2, Fc = 3θ and Fc 2 = 8θ 2. Then (9) boils down to Fc 2 (Fc) 2 = 9θ 2 3 Statistical manifold 4(5e 3) θ2.4 θ 2. (23) The well-known example of a family with optimal estimator is the exponential family with One sees immediately that p θ (x) = exp (G(θ) θ k c k (x)) (24) G(θ) = log dµ(x) e θk c k (x). (25) θ k p θ(x) = p θ (x) ( ) θ G(θ) c k(x), (26) k which is (7) with Z(θ) identically and the escort pdf P θ equal to p θ. This example motivates also the geometric interpretation of (), in the form (7), as a linear map between tangent planes. The score variables log p θ /θ k of the standard statistical manifold are replaced by the variables P θ (x) θ kp θ(x). (27) They are tangent vectors of the concave function G(θ) θ l c l. The metric tensor of the latter function is a constant random variable. The geometry of the manifold of random variables (G(θ) θ l c l ) θ D is transferred onto the family of pdfs (p θ ) θ D. Note that the score variables have vanishing expectation F θ. It is now obvious to define an inner product of random variables by A, B θ = F θ AB. (28) 6
7 Then one has p θ P θ θ, p θ k P θ θ l θ = g kl (θ). (29) Let g kl (θ) denote the inverse of g kl (θ) (assume it exists). Then a projection operator π θ onto the orthogonal complement of the tangent plane is defined by π θ A = A g kl p θ P θ θ, A k θ P θ p θ θ l F θ A. (3) If () is satisfied, then [ ] p θ Z π θ = π θ l P θ θ k θ θ (F θc l k c k ) + Z(θ) 2 G θ k θ l = Z [ F θ l θ c k c k + g lm (θ) ] p θ P θ θ, c p θ k l θ P θ θ m = Z [ ] p θ F θ l θ c k c k Z(θ)P θ θ k =. (3) This follows also immediately from p θ θ l P θ θ = k Z(θ) Z p θ θ l P θ θ + Z(θ) 2 G k θ k θl. (32) That the derivatives of the score variables are linear combinations of the score variables and the constant random variable is usually the characteristic feature of the exponential family. This is a motivation to introduce a generalized notion of exponential family. 4 φ-logarithms and φ-exponentials In the next section the notion of exponential family is generalized to a rather large class of families of pdfs. This is done by replacing the exponential function by some other function satisfying a minimal number of requirements. The latter function will be called a deformed exponential and will be denoted exp φ. This has the advantage that the resulting expressions look very familiar, resembling those of the exponential family. Fix an increasing function φ of [, + ), strictly positive on (, + ). It is used to define the φ-logarithm ln φ by ln φ (u) = u dv, u >. (33) φ(v) 7
8 Clearly, ln φ is a concave function which is negative on (, ) and positive on (, + ). The inverse of the function ln φ is denoted exp φ. It is defined on the range of ln φ. The definition can be extended to all of R by putting exp φ (u) = if u is too small and exp φ = + if u is too large. In case φ(u) = u for all u then ln φ coincides with the natural logarithm and exp φ coincides with the exponential function. Given φ, introduce a function ψ of R by ψ(u) = φ(exp φ (u)) if u is in the range of ln φ = if u is too small = + if u is too large. (34) Clearly is φ(u) = ψ(ln φ (u)) for all u >. Proposition One has for all u in R exp φ (u) = + = u u dv ψ(v) dv ψ(v) +. (35) Proof First consider the case that [, u) belongs to the range of ln φ. Then a substitution of integration variables v = ln φ (w) is possible. One finds, using dv/dw = /φ(w) and ψ(v) = φ( exp φ (v)) = φ(w), u dv ψ(v) = expφ (u) dw = exp φ (u). (36) Using exp φ ( ) = one concludes (35). In case M = sup v ln φ (v) is finite and u M then ψ(v) = + for v [M, u]. One has u dv ψ(v) = M + dv ψ(v) dw = +. (37) But also the l.h.s. of (35) is infinite. Hence the equality holds. Finally, if m = inf v ln φ (v) is finite and u m then ψ(v) = holds for v m. Hence u dv ψ(v) = 8 m dv ψ(v)
9 This ends the proof. = dw =. (38) Proposition 2 The function exp φ is continuous on the open interval of points where it does not diverge. Proof Let m and M be as in the proof of the previous proposition. Then exp φ is differentiable on (m, M). If m = this ends the proof. If m is finite then it suffices to verify that exp φ (u) is continuous in u = m. But this is straightforward. Example 2 Let φ(u) = u q with q >. This function is increasing and strictly positive on (, + ). Hence, it defines a φ-logarithm which will be denoted ln q and is given by ln q (u) = u dv v q = u q q if q = log(u) if q =. (39) This deformed logarithm has been introduced in the context of nonextensive statistical physics in []. The inverse function is denoted exp q and is given by The function ψ is then given by exp q (u) = [ + ( q)u] /( q) +. (4) ψ(u) = [ + ( q)u] q/( q) +. (4) Example 3 Let φ(x) = x, the smallest integer not smaller than x. This piecewise constant function is increasing and strictly positive on (, + ). Hence, ln φ is piecewise linear. The function ψ is given by ψ(x) = if x = φ( + x) otherwise. (42) The φ-exponential exp φ is also piecewise linear and satisfies exp φ (x) = x. (43) 9
10 5 The φ-exponential family Let φ be given as in the previous section. Fix a measure space, µ and a set of random variables c k, k =,, n. The φ-exponential family of pdfs (p θ ) θ D is defined by p θ (x) = exp φ (G(θ) θ k c k (x)). (44) The domain D is an open set of θ for which G(θ) exists such that (44) is properly normalized, i.e. p θ M (µ). The distributions (44) are the equilibrium pdfs of generalized thermostatistics as introduced in [2, 3]. Proposition 3 The function G(θ) is concave on D. Proof Assume θ, η and λθ + ( λ)η in D for some λ in [, ]. Then, using convexity of exp φ, Hence exp φ (λg(θ) + ( λ)g(η) [λθ k + ( λ)η k ]c k (x)) λp θ (x) + ( λ)p η (x). (45) R n dµ(x) exp φ ( λg(θ) + ( λ)g(η) [λθ k + ( λ)ηk]c k (x) ). (46) Since exp φ is increasing one concludes that λg(θ) + ( λ)g(η) G(λθ + ( λ)η). (47) Hence G is concave. Proposition 4 Let ψ be determined by φ via (34). If the integral Z(θ) = dµ(x) ψ(g(θ) θ k c k (x)) (48) converges for all θ D, then (p θ ) θ D has an escort family (P θ ) θ D, given by Condition () is satisfied. P θ (x) = Z(θ) φ(p θ(x)) if p θ (x) > = otherwise. (49)
11 Proof One has φ(p θ (x)) = φ(exp φ (G(θ) θ k c k (x))) = ψ(g(θ) θ k c k (x)). (5) Because φ(p θ (x)) cannot be zero for µ-almost all x one concludes that Z(θ) > and that P θ is properly normalized. From the properties of the function exp φ follows immediately that θ lp θ(x) = ψ(g(θ) θ k c k (x)) θ l (G(θ) θm c m (x)) = Z(θ)P θ (x) θ l(g(θ) θm c m (x)). (5) This proves that (P θ ) θ D satisfies (). Example 2 continued Let φ(u) = u q as in Example 2 above. The pdfs p θ are given by p θ (x) = [ + ( q)(g(θ) θ k c k (x)) ] /( q) + for θ in a suitable domain D. The escort probabilities are with P θ (x) = Z(θ) =, (52) [ + ( q)(g(θ) θ k c k (x)) ] q/( q) (53) Z(θ) + dµ(x) [ + ( q)(g(θ) θ k c k (x)) ] q/( q) + (54) (assuming convergence of these integrals). The family (p θ ) θ D coincides with Amari s α-family [3], with α given by α = 2q. Example continued Example is the q = -limit of example 2. Let φ(u) = for all u >. Then ln φ (u) = u exp φ (u) = [ + u] + ψ(u) = if u > ; = otherwise. (55) One has p θ (x) = 2 θ [ x ] θ +
12 ( 2 = exp φ θ 2x ). (56) θ 2 This is a φ-exponential family with parameter Θ = /θ 2, estimator c(x) = 2x and scale function G(Θ) = 2 Θ. The escort probabilities, making inequality (9) optimally satisfied, are given by P Θ (x) = θ I x θ. (57) The information matrix g(θ) equals θ 4 /3. Further is F Θ c = θ and F Θ c 2 = 4θ 2 /3 and Θ F(Θ) = E Θc = 2θ/3 = 2/3 Θ. (58) It is now straightforward to verify that the inequality (9) is optimally satisfied. 6 Divergences Divergences of the Bregman type are needed for what follows. In the form given below they have been introduced in [4]. Fix a strictly positive increasing function φ of [, + ). Introduce p(x) D φ (p p ) = dµ(x) du [ln φ (u) ln φ (p (x))]. (59) p (x) D φ (p p ) follows because ln φ is an increasing function. Also convexity in the first argument follows because ln φ is an increasing function. Let (p θ ) θ D be φ-exponential. Then infinitesimal variation of the divergence D φ (p p ) reproduces the metric tensor g kl (θ), up to a scalar function. Indeed, one has θ kd φ(p θ p η ) η=θ = η kd φ(p θ p η ) η=θ = (6) and 2 θ k θ ld φ(p θ p η ) η=θ = dµ(x) [ln θ k φ (p θ (x)) ln φ (p η (x))] θ lp θ(x) 2 η=θ
13 = = [ ][ ] dµ(x) φ(p θ (x)) θ p θ(x) k θ lp θ(x) Z(θ) g kl(θ). (6) Similar calculations give 2 θ k η ld φ(p θ p η ) = η=θ 2 η k η ld φ(p θ p η ) = η=θ Z(θ) g kl(θ). (62) 7 Information content In [5] the definition of deformed logarithm contains the additional condition that the integral du ln φ (u) = du u < + (63) φ(u) converges. This condition is needed in the definition of entropy functional / information content based on the deformed logarithm. Introduce another strictly increasing positive function χ by χ(v) = [ /v ] du u (64) φ(u) The motivation for introducing this function comes from the fact that it satisfies the following property. Lemma d dv v ln χ(/v) = ln φ (v) du u φ(u). (65) Proof d dv v ln χ(/v) = ln χ (/v) vχ(/v) /v = du χ(u) v du u v φ(u) /v /u z = du dz φ(z) v du u v φ(u) v = du u z dz u 2 φ(z) v du u v φ(u) 3
14 = dz z φ(z) ln φ(v), (66) which is the desired result. Define information content (also called entropy functional) I φ (p) of a pdf p in M (µ) by I φ (p) = dµ(x) p(x) ln χ (/p(x)) (67) whenever the integral converges. Using the lemma one verifies immediately that I φ (p) is a concave function of p. A short calculation gives /p(x) I φ (p) = dµ(x) p(x) du χ(u) p(x) = dµ(x) p(x) χ(/v) d v p(x) [ v = dµ(x) p(x) du u ] d [ φ(u) v = dµ(x) p(x) p(x)χ(/p(x)) ] ln φ(p(x)) χ() = p(x) χ() dµ(x) du ln φ (u). (68) This implies that I φ (p) I φ (p ) = dµ(x) and hence D φ (p p ) = I φ (p ) I φ (p) p(x) p (x) du ln φ (u), (69) dµ(x) (p(x) p (x)) ln φ (p (x)). (7) This relation links the divergence D φ (p p ) with the information function I φ (p). The following result shows that the φ-exponential family is a conditional maximizer of I φ. It also shows that the scale function F is the Legendre transform of the information content I φ 4
15 Theorem 2 Let (p θ ) θ D be φ-exponential, with estimator c and scale functions F and G. Then there exists a constant F such that F(θ) = F + min {E p θ k c k I φ (p)}. (7) p M (µ) The minimum is attained for p = p θ. In particular, F(θ) is a convex function of θ and p θ maximizes I φ (p) under the constraint that Proof Let us first show that for any pdf p E p θ k c k = E θ θ k c k. (72) E p θ k c k I φ (p) E θ θ k c k I φ (p θ ). (73) One has dµ(x) (p(x) p θ (x)) ln φ (p θ (x)) = dµ(x) (p(x) p θ (x)) [ ] G(θ) θ k c k = (E p E θ )θ k c k. (74) Hence, (7) becomes now D φ (p p θ ) = I φ (p θ ) I φ (p) + (E p E θ )θ k c k. (75) But one has always D φ (p p θ ). Therefore, (73) follows. Next calculate, using the lemma, ( θ ki φ(p θ ) = dµ(x) ln φ (p θ (x)) du u ) φ(u) θ p θ(x) ( k = dµ(x) G(θ) + θ l c l (x) du u ) φ(u) θ p θ(x) k = dµ(x) ( θ l c l (x) ) θ p θ(x) k = θ k ( Eθ θ l c l ) Eθ c k. (76) Because c is an estimator with scale function F one obtains θ k ( Eθ θ l c l I φ (p θ ) ) = Hence there exists a constant F for which F(θ). (77) θk F(θ) = F + E θ θ l c l I φ (p θ ). (78) 5
16 In combination with (73) this results in (7). Without restriction one can assume F =. In statistical physics the function F(θ) is the free energy as a function of temperature, up to some proportionality factor. Example 4 Let φ(u) = u 2 q /q, with < q < 2. This is of course only a reparametrization of example 2, which is done to recover expressions found in the literature. The deformed logarithm is given by q ln φ (u) = q (uq ) if q = log(u) if q =. (79) One obtains χ(v) = v q and hence I φ (p) = dµ(x) p(x) p(x)q. (8) q This is the entropy functional proposed by Tsallis [2] as a basis for nonextensive thermostatistics, and reported earlier in the literature by Havrda and Charvat [6] and by Daróczy [7]. The corresponding expression for the divergence is D φ (p, p ) = dµ(x) p(x) [ p(x) q p (x) q ] q dµ(x) [p(x) p (x)] p (x) q. (8) 8 Dual coordinates Introduce dual coordinates Assume () holds. Then, one obtains from (9) η k = E θ c k = F θk. (82) η k = θ l θ E θc l k 2 = θ l θ kf(θ) = Z(θ) [F θ c k c l (F θ c k )(F θ c l )] 6
17 = Z(θ) g kl(θ). (83) To obtain the last line a φ-exponential family has been assumed. This relation implies θ k η l = Z(θ)g kl (θ). (84) Thes are the orthogonality relations between the two sets of coordinates θ and η. Next we derive the dual relation of (82). Proposition 5 Let (p θ ) θ D be φ-exponential. Assume the regularity condition (8) is satisfied. Then θ k = η k I φ (p θ ). (85) Proof One calculates (assume integration and partial derivative can be interchanged), using Lemma (), [ θ ki φ(p θ ) = dµ(x) ln φ (p θ (x)) + du u ] φ(u) θ p θ(x) [ k = dµ(x) G(θ) θ l c l (x) + du u ] φ(u) θ p θ(x) k = dµ(x) θ l c l (x) θ p θ(x). (86) k To obtain the last line the regularity condition has been used. Use now that p θ satisfies (). One obtains In combination with (84) this gives θ ki φ(p θ ) = Z(θ)F θ θ l c l (F θ c k c k ) = Z(θ)θ l g lk (θ). (87) η l I φ (p θ ) = ( ) θ k θ li φ(p θ ) η l ( = ( Z(θ)θ m g ml (θ)) ) Z(θ) gkl (θ) = θ l. (88) 7
18 Equation (85) is the dual relation of (82). Expression (78) can now be written as with E(η) = I φ (p θ ). F(θ) + E(η) = θ k η k (89) 9 Discussion The present paper introduces generalized exponential families, and calls them φ-exponential because they depend on the choice of a strictly positive nondecreasing function φ of (, + ). Several properties, known to hold for the exponential family, can be generalized. The paper starts with a generalization of the well-known lower bound of Cramer and Rao, involving the concept of escort probability distributions. It is shown that the φ-exponential family optimizes this generalized lower bound. The metric tensor, which generalizes the Fisher information, depends on both the family of pdfs and the escort family, and determines the geometry of the statistical manifold. In the final part of the paper deals with the dual structure of the statistical manifold. In statistical physics this duality maps mean energy onto temperature and entropy onto free energy. The dual structure is shown to exist in the general context of φ-exponential families. Throughout the paper the number of parameters n has been assumed to be finite. A non-parametrized approach to statistical manifolds is found in [8]. The extension of the present work to this more abstract context has not been considered. Acknowledgments I am thankful to S. Abe who urged me to study the geometry of statistical distributions. I thank Dr. Ch. Vignat for pointing out ref. [8]. References [] J.W. Gibbs, Elementary principles in statistical mechanics developed with special reference to the rational foundation of thermodynamics (Dover, 96) [2] C. Tsallis, Possible Generalization of Boltzmann-Gibbs Statistics, J. Stat. Phys. 52, (988). 8
19 [3] S. Amari, Differential-geometrical methods in statistics, Lecture Notes in Statistics 28 (985). [4] M.K. Murray, J.W. Rice, Differential geometry and statistics (Chapman and Hall, 993) [5] C. Beck, F. Schlögl, Thermodynamics of chaotic systems: An introduction (Cambridge University Press, Cambridge, 993) [6] R. Trasarti-Battistoni, Euclidean and Riemannian geometrical approaches to non-extensive thermo-statistical mechanics, arxiv:cond-mat/ [7] S. Abe, q-deformed Entropies and Fisher Metrics, in: Proceedings of The 5th International Wigner Symposium, (August 25-29, 997, Vienna, Austria), eds. P. Kasperkovitz and D. Grau (World Scientific, Singapore, 998) p. 66. [8] C. Tsallis, Generalized entropy-based criterion for consistent testing, Phys. Rev. E58(2), (998). [9] M. Shiino, H-theorem with generalized relative entropies and the Tsallis statistics, J. Phys. Soc. Jpn 67(), (998). [] S. Abe, Geometry of escort distributions, arxiv:cond-mat/3523, Phys. Rev. 68, 3 (23). [] C. Tsallis, What are the numbers that experiments provide? Quimica Nova 7, 468 (994). [2] J. Naudts, Generalized thermostatistics and mean-field theory, arxiv::cond-mat/2444, Physica A332, (24). [3] J. Naudts, Generalized thermostatistics based on deformed exponential and logarithmic functions, arxiv::cond-mat/3438. [4] J. Naudts, Continuity of a class of entropies and relative entropies, arxiv::cond-mat/2838. [5] J Naudts, Deformed exponentials and logarithms in generalized thermostatistics, arxiv::cond-mat/23489, Physica A36, (22). [6] J. Havrda, F. Charvat, Kybernetica 3, 3-35 (967). [7] Z. Daróczy, Inform. Control 6, 36- (97). [8] G. Pistone, C. Sempi, An infinite-dimensional geometric structure on the space of all the probability measures equivalent to a given one, Ann. Statist. 23, (995). 9
Statistical physics models belonging to the generalised exponential family
Statistical physics models belonging to the generalised exponential family Jan Naudts Universiteit Antwerpen 1. The generalized exponential family 6. The porous media equation 2. Theorem 7. The microcanonical
More informationDually Flat Geometries in the State Space of Statistical Models
1/ 12 Dually Flat Geometries in the State Space of Statistical Models Jan Naudts Universiteit Antwerpen ECEA, November 2016 J. Naudts, Dually Flat Geometries in the State Space of Statistical Models. In
More informationThe q-exponential family in statistical physics
Journal of Physics: Conference Series The q-exponential family in statistical physics To cite this article: Jan Naudts 00 J. Phys.: Conf. Ser. 0 0003 View the article online for updates and enhancements.
More informationarxiv: v1 [math-ph] 29 Apr 2016
INFORMATION THEORY AND STATISTICAL MECHANICS REVISITED arxiv:64.8739v [math-h] 29 Ar 26 JIAN ZHOU Abstract. We derive Bose-Einstein statistics and Fermi-Dirac statistics by Princile of Maximum Entroy alied
More informationEntropy measures of physics via complexity
Entropy measures of physics via complexity Giorgio Kaniadakis and Flemming Topsøe Politecnico of Torino, Department of Physics and University of Copenhagen, Department of Mathematics 1 Introduction, Background
More informationInformation geometry of Bayesian statistics
Information geometry of Bayesian statistics Hiroshi Matsuzoe Department of Computer Science and Engineering, Graduate School of Engineering, Nagoya Institute of Technology, Nagoya 466-8555, Japan Abstract.
More informationInformation geometry of mirror descent
Information geometry of mirror descent Geometric Science of Information Anthea Monod Department of Statistical Science Duke University Information Initiative at Duke G. Raskutti (UW Madison) and S. Mukherjee
More informationInformation Geometry and Algebraic Statistics
Matematikos ir informatikos institutas Vinius Information Geometry and Algebraic Statistics Giovanni Pistone and Eva Riccomagno POLITO, UNIGE Wednesday 30th July, 2008 G.Pistone, E.Riccomagno (POLITO,
More informationChapter 2 Exponential Families and Mixture Families of Probability Distributions
Chapter 2 Exponential Families and Mixture Families of Probability Distributions The present chapter studies the geometry of the exponential family of probability distributions. It is not only a typical
More informationarxiv:cond-mat/ v1 10 Aug 2002
Model-free derivations of the Tsallis factor: constant heat capacity derivation arxiv:cond-mat/0208205 v1 10 Aug 2002 Abstract Wada Tatsuaki Department of Electrical and Electronic Engineering, Ibaraki
More information40.530: Statistics. Professor Chen Zehua. Singapore University of Design and Technology
Singapore University of Design and Technology Lecture 9: Hypothesis testing, uniformly most powerful tests. The Neyman-Pearson framework Let P be the family of distributions of concern. The Neyman-Pearson
More informationStatistics 612: L p spaces, metrics on spaces of probabilites, and connections to estimation
Statistics 62: L p spaces, metrics on spaces of probabilites, and connections to estimation Moulinath Banerjee December 6, 2006 L p spaces and Hilbert spaces We first formally define L p spaces. Consider
More informationTutorial: Statistical distance and Fisher information
Tutorial: Statistical distance and Fisher information Pieter Kok Department of Materials, Oxford University, Parks Road, Oxford OX1 3PH, UK Statistical distance We wish to construct a space of probability
More informationSYMPLECTIC GEOMETRY: LECTURE 5
SYMPLECTIC GEOMETRY: LECTURE 5 LIAT KESSLER Let (M, ω) be a connected compact symplectic manifold, T a torus, T M M a Hamiltonian action of T on M, and Φ: M t the assoaciated moment map. Theorem 0.1 (The
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationJune 21, Peking University. Dual Connections. Zhengchao Wan. Overview. Duality of connections. Divergence: general contrast functions
Dual Peking University June 21, 2016 Divergences: Riemannian connection Let M be a manifold on which there is given a Riemannian metric g =,. A connection satisfying Z X, Y = Z X, Y + X, Z Y (1) for all
More informationAn introduction to some aspects of functional analysis
An introduction to some aspects of functional analysis Stephen Semmes Rice University Abstract These informal notes deal with some very basic objects in functional analysis, including norms and seminorms
More informationF -Geometry and Amari s α Geometry on a Statistical Manifold
Entropy 014, 16, 47-487; doi:10.3390/e160547 OPEN ACCESS entropy ISSN 1099-4300 www.mdpi.com/journal/entropy Article F -Geometry and Amari s α Geometry on a Statistical Manifold Harsha K. V. * and Subrahamanian
More information1. Fisher Information
1. Fisher Information Let f(x θ) be a density function with the property that log f(x θ) is differentiable in θ throughout the open p-dimensional parameter set Θ R p ; then the score statistic (or score
More informationMarch 10, 2017 THE EXPONENTIAL CLASS OF DISTRIBUTIONS
March 10, 2017 THE EXPONENTIAL CLASS OF DISTRIBUTIONS Abstract. We will introduce a class of distributions that will contain many of the discrete and continuous we are familiar with. This class will help
More informationConjugate Predictive Distributions and Generalized Entropies
Conjugate Predictive Distributions and Generalized Entropies Eduardo Gutiérrez-Peña Department of Probability and Statistics IIMAS-UNAM, Mexico Padova, Italy. 21-23 March, 2013 Menu 1 Antipasto/Appetizer
More informationAN AFFINE EMBEDDING OF THE GAMMA MANIFOLD
AN AFFINE EMBEDDING OF THE GAMMA MANIFOLD C.T.J. DODSON AND HIROSHI MATSUZOE Abstract. For the space of gamma distributions with Fisher metric and exponential connections, natural coordinate systems, potential
More information6.1 Variational representation of f-divergences
ECE598: Information-theoretic methods in high-dimensional statistics Spring 2016 Lecture 6: Variational representation, HCR and CR lower bounds Lecturer: Yihong Wu Scribe: Georgios Rovatsos, Feb 11, 2016
More informationRelative Loss Bounds for Multidimensional Regression Problems
Relative Loss Bounds for Multidimensional Regression Problems Jyrki Kivinen and Manfred Warmuth Presented by: Arindam Banerjee A Single Neuron For a training example (x, y), x R d, y [0, 1] learning solves
More informationHOPF S DECOMPOSITION AND RECURRENT SEMIGROUPS. Josef Teichmann
HOPF S DECOMPOSITION AND RECURRENT SEMIGROUPS Josef Teichmann Abstract. Some results of ergodic theory are generalized in the setting of Banach lattices, namely Hopf s maximal ergodic inequality and the
More informationarxiv: v1 [cs.lg] 1 May 2010
A Geometric View of Conjugate Priors Arvind Agarwal and Hal Daumé III School of Computing, University of Utah, Salt Lake City, Utah, 84112 USA {arvind,hal}@cs.utah.edu arxiv:1005.0047v1 [cs.lg] 1 May 2010
More informationLECTURE 15: COMPLETENESS AND CONVEXITY
LECTURE 15: COMPLETENESS AND CONVEXITY 1. The Hopf-Rinow Theorem Recall that a Riemannian manifold (M, g) is called geodesically complete if the maximal defining interval of any geodesic is R. On the other
More informationLecture 7 Introduction to Statistical Decision Theory
Lecture 7 Introduction to Statistical Decision Theory I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 20, 2016 1 / 55 I-Hsiang Wang IT Lecture 7
More informationPower law distribution of Rényi entropy for equilibrium systems having nonadditive energy
arxiv:cond-mat/030446v5 [cond-mat.stat-mech] 0 May 2003 Power law distribution of Rényi entropy for equilibrium systems having nonadditive energy Qiuping A. Wang Institut Supérieur des Matériaux du Mans,
More informationFunctional Analysis. Franck Sueur Metric spaces Definitions Completeness Compactness Separability...
Functional Analysis Franck Sueur 2018-2019 Contents 1 Metric spaces 1 1.1 Definitions........................................ 1 1.2 Completeness...................................... 3 1.3 Compactness......................................
More informationA Few Notes on Fisher Information (WIP)
A Few Notes on Fisher Information (WIP) David Meyer dmm@{-4-5.net,uoregon.edu} Last update: April 30, 208 Definitions There are so many interesting things about Fisher Information and its theoretical properties
More informationConvex Functions. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Convex Functions Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Definition convex function Examples
More informationRobustness and duality of maximum entropy and exponential family distributions
Chapter 7 Robustness and duality of maximum entropy and exponential family distributions In this lecture, we continue our study of exponential families, but now we investigate their properties in somewhat
More informationEXPOSITORY NOTES ON DISTRIBUTION THEORY, FALL 2018
EXPOSITORY NOTES ON DISTRIBUTION THEORY, FALL 2018 While these notes are under construction, I expect there will be many typos. The main reference for this is volume 1 of Hörmander, The analysis of liner
More informationarxiv:cond-mat/ v1 8 Jan 2004
Multifractality and nonextensivity at the edge of chaos of unimodal maps E. Mayoral and A. Robledo arxiv:cond-mat/0401128 v1 8 Jan 2004 Instituto de Física, Universidad Nacional Autónoma de México, Apartado
More informationReview and continuation from last week Properties of MLEs
Review and continuation from last week Properties of MLEs As we have mentioned, MLEs have a nice intuitive property, and as we have seen, they have a certain equivariance property. We will see later that
More informationarxiv: v2 [stat.ot] 25 Oct 2017
A GEOMETER S VIEW OF THE THE CRAMÉR-RAO BOUND ON ESTIMATOR VARIANCE ANTHONY D. BLAOM arxiv:1710.01598v2 [stat.ot] 25 Oct 2017 Abstract. The classical Cramér-Rao inequality gives a lower bound for the variance
More informationA Geometric View of Conjugate Priors
Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence A Geometric View of Conjugate Priors Arvind Agarwal Department of Computer Science University of Maryland College
More informationChapter 3 : Likelihood function and inference
Chapter 3 : Likelihood function and inference 4 Likelihood function and inference The likelihood Information and curvature Sufficiency and ancilarity Maximum likelihood estimation Non-regular models EM
More informationSpectral Gap and Concentration for Some Spherically Symmetric Probability Measures
Spectral Gap and Concentration for Some Spherically Symmetric Probability Measures S.G. Bobkov School of Mathematics, University of Minnesota, 127 Vincent Hall, 26 Church St. S.E., Minneapolis, MN 55455,
More informationDeviation Measures and Normals of Convex Bodies
Beiträge zur Algebra und Geometrie Contributions to Algebra Geometry Volume 45 (2004), No. 1, 155-167. Deviation Measures Normals of Convex Bodies Dedicated to Professor August Florian on the occasion
More informationPICARD S THEOREM STEFAN FRIEDL
PICARD S THEOREM STEFAN FRIEDL Abstract. We give a summary for the proof of Picard s Theorem. The proof is for the most part an excerpt of [F]. 1. Introduction Definition. Let U C be an open subset. A
More informationLecture 4 Lebesgue spaces and inequalities
Lecture 4: Lebesgue spaces and inequalities 1 of 10 Course: Theory of Probability I Term: Fall 2013 Instructor: Gordan Zitkovic Lecture 4 Lebesgue spaces and inequalities Lebesgue spaces We have seen how
More information1 Differentiable manifolds and smooth maps
1 Differentiable manifolds and smooth maps Last updated: April 14, 2011. 1.1 Examples and definitions Roughly, manifolds are sets where one can introduce coordinates. An n-dimensional manifold is a set
More informationOptimization and Optimal Control in Banach Spaces
Optimization and Optimal Control in Banach Spaces Bernhard Schmitzer October 19, 2017 1 Convex non-smooth optimization with proximal operators Remark 1.1 (Motivation). Convex optimization: easier to solve,
More informationQuantum Fisher Information. Shunlong Luo Beijing, Aug , 2006
Quantum Fisher Information Shunlong Luo luosl@amt.ac.cn Beijing, Aug. 10-12, 2006 Outline 1. Classical Fisher Information 2. Quantum Fisher Information, Logarithm 3. Quantum Fisher Information, Square
More informationECE 275A Homework 7 Solutions
ECE 275A Homework 7 Solutions Solutions 1. For the same specification as in Homework Problem 6.11 we want to determine an estimator for θ using the Method of Moments (MOM). In general, the MOM estimator
More informationCharacterization of Self-Polar Convex Functions
Characterization of Self-Polar Convex Functions Liran Rotem School of Mathematical Sciences, Tel-Aviv University, Tel-Aviv 69978, Israel Abstract In a work by Artstein-Avidan and Milman the concept of
More informationarxiv:math-ph/ v1 30 May 2005
Nongeneralizability of Tsallis Entropy by means of Kolmogorov-Nagumo averages under pseudo-additivity arxiv:math-ph/0505078v1 30 May 2005 Ambedkar Dukkipati, 1 M. Narsimha Murty,,2 Shalabh Bhatnagar 3
More informationA Geometric View of Conjugate Priors
A Geometric View of Conjugate Priors Arvind Agarwal and Hal Daumé III School of Computing, University of Utah, Salt Lake City, Utah, 84112 USA {arvind,hal}@cs.utah.edu Abstract. In Bayesian machine learning,
More informationPower law distribution of Rényi entropy for equilibrium systems having nonadditive energy
arxiv:cond-mat/0304146v1 [cond-mat.stat-mech] 7 Apr 2003 Power law distribution of Rényi entropy for equilibrium systems having nonadditive energy Qiuping A. Wang Institut Supérieur des Matériaux du Mans,
More informationConvexity/Concavity of Renyi Entropy and α-mutual Information
Convexity/Concavity of Renyi Entropy and -Mutual Information Siu-Wai Ho Institute for Telecommunications Research University of South Australia Adelaide, SA 5095, Australia Email: siuwai.ho@unisa.edu.au
More informationtopics about f-divergence
topics about f-divergence Presented by Liqun Chen Mar 16th, 2018 1 Outline 1 f-gan: Training Generative Neural Samplers using Variational Experiments 2 f-gans in an Information Geometric Nutshell Experiments
More informationCS Lecture 19. Exponential Families & Expectation Propagation
CS 6347 Lecture 19 Exponential Families & Expectation Propagation Discrete State Spaces We have been focusing on the case of MRFs over discrete state spaces Probability distributions over discrete spaces
More informationarxiv:math-ph/ v4 22 Mar 2005
Nonextensive triangle euality and other properties of Tsallis relative-entropy minimization arxiv:math-ph/0501025v4 22 Mar 2005 Ambedkar Dukkipati, 1 M. Narasimha Murty,,2 Shalabh Bhatnagar 3 Department
More informationBayesian statistics: Inference and decision theory
Bayesian statistics: Inference and decision theory Patric Müller und Francesco Antognini Seminar über Statistik FS 28 3.3.28 Contents 1 Introduction and basic definitions 2 2 Bayes Method 4 3 Two optimalities:
More information2 Lebesgue integration
2 Lebesgue integration 1. Let (, A, µ) be a measure space. We will always assume that µ is complete, otherwise we first take its completion. The example to have in mind is the Lebesgue measure on R n,
More informationU Logo Use Guidelines
Information Theory Lecture 3: Applications to Machine Learning U Logo Use Guidelines Mark Reid logo is a contemporary n of our heritage. presents our name, d and our motto: arn the nature of things. authenticity
More informationOpen Questions in Quantum Information Geometry
Open Questions in Quantum Information Geometry M. R. Grasselli Department of Mathematics and Statistics McMaster University Imperial College, June 15, 2005 1. Classical Parametric Information Geometry
More informationCourse 212: Academic Year Section 1: Metric Spaces
Course 212: Academic Year 1991-2 Section 1: Metric Spaces D. R. Wilkins Contents 1 Metric Spaces 3 1.1 Distance Functions and Metric Spaces............. 3 1.2 Convergence and Continuity in Metric Spaces.........
More informationLogarithmic Sobolev Inequalities
Logarithmic Sobolev Inequalities M. Ledoux Institut de Mathématiques de Toulouse, France logarithmic Sobolev inequalities what they are, some history analytic, geometric, optimal transportation proofs
More informationNONCLASSICAL SHOCK WAVES OF CONSERVATION LAWS: FLUX FUNCTION HAVING TWO INFLECTION POINTS
Electronic Journal of Differential Equations, Vol. 2006(2006), No. 149, pp. 1 18. ISSN: 1072-6691. URL: http://ejde.math.txstate.edu or http://ejde.math.unt.edu ftp ejde.math.txstate.edu (login: ftp) NONCLASSICAL
More informationλ(x + 1)f g (x) > θ 0
Stat 8111 Final Exam December 16 Eleven students took the exam, the scores were 92, 78, 4 in the 5 s, 1 in the 4 s, 1 in the 3 s and 3 in the 2 s. 1. i) Let X 1, X 2,..., X n be iid each Bernoulli(θ) where
More informationLegendre-Fenchel transforms in a nutshell
1 2 3 Legendre-Fenchel transforms in a nutshell Hugo Touchette School of Mathematical Sciences, Queen Mary, University of London, London E1 4NS, UK Started: July 11, 2005; last compiled: August 14, 2007
More informationarxiv: v2 [math.pr] 7 Nov 2007
arxiv:0707.4273v2 [math.pr] 7 Nov 2007 A convexity property of expectations under exponential weights M. Balázs balazs@math.bme.hu T. Seppäläinen seppalai@math.wisc.edu October 29, 208 Abstract Please
More informationSymplectic and Kähler Structures on Statistical Manifolds Induced from Divergence Functions
Symplectic and Kähler Structures on Statistical Manifolds Induced from Divergence Functions Jun Zhang 1, and Fubo Li 1 University of Michigan, Ann Arbor, Michigan, USA Sichuan University, Chengdu, China
More informationLecture 35: December The fundamental statistical distances
36-705: Intermediate Statistics Fall 207 Lecturer: Siva Balakrishnan Lecture 35: December 4 Today we will discuss distances and metrics between distributions that are useful in statistics. I will be lose
More informationMath 598 Feb 14, Geometry and Topology II Spring 2005, PSU
Math 598 Feb 14, 2005 1 Geometry and Topology II Spring 2005, PSU Lecture Notes 7 2.7 Smooth submanifolds Let N be a smooth manifold. We say that M N m is an n-dimensional smooth submanifold of N, provided
More informationBanach Spaces V: A Closer Look at the w- and the w -Topologies
BS V c Gabriel Nagy Banach Spaces V: A Closer Look at the w- and the w -Topologies Notes from the Functional Analysis Course (Fall 07 - Spring 08) In this section we discuss two important, but highly non-trivial,
More informationSmooth Submanifolds Intersecting any Analytic Curve in a Discrete Set
Syracuse University SURFACE Mathematics Faculty Scholarship Mathematics 2-23-2004 Smooth Submanifolds Intersecting any Analytic Curve in a Discrete Set Dan Coman Syracuse University Norman Levenberg University
More information(1) is an invertible sheaf on X, which is generated by the global sections
7. Linear systems First a word about the base scheme. We would lie to wor in enough generality to cover the general case. On the other hand, it taes some wor to state properly the general results if one
More informationChapter 7. Confidence Sets Lecture 30: Pivotal quantities and confidence sets
Chapter 7. Confidence Sets Lecture 30: Pivotal quantities and confidence sets Confidence sets X: a sample from a population P P. θ = θ(p): a functional from P to Θ R k for a fixed integer k. C(X): a confidence
More informationSOLUTION FOR HOMEWORK 7, STAT p(x σ) = (1/[2πσ 2 ] 1/2 )e (x µ)2 /2σ 2.
SOLUTION FOR HOMEWORK 7, STAT 6332 1. We have (for a general case) Denote p (x) p(x σ)/ σ. Then p(x σ) (1/[2πσ 2 ] 1/2 )e (x µ)2 /2σ 2. p (x σ) p(x σ) 1 (x µ)2 +. σ σ 3 Then E{ p (x σ) p(x σ) } σ 2 2σ
More informationConvergent Iterative Algorithms in the 2-inner Product Space R n
Int. J. Open Problems Compt. Math., Vol. 6, No. 4, December 2013 ISSN 1998-6262; Copyright c ICSRS Publication, 2013 www.i-csrs.org Convergent Iterative Algorithms in the 2-inner Product Space R n Iqbal
More informationA View on Extension of Utility-Based on Links with Information Measures
Communications of the Korean Statistical Society 2009, Vol. 16, No. 5, 813 820 A View on Extension of Utility-Based on Links with Information Measures A.R. Hoseinzadeh a, G.R. Mohtashami Borzadaran 1,b,
More information3 Integration and Expectation
3 Integration and Expectation 3.1 Construction of the Lebesgue Integral Let (, F, µ) be a measure space (not necessarily a probability space). Our objective will be to define the Lebesgue integral R fdµ
More informationLecture 5: Linear models for classification. Logistic regression. Gradient Descent. Second-order methods.
Lecture 5: Linear models for classification. Logistic regression. Gradient Descent. Second-order methods. Linear models for classification Logistic regression Gradient descent and second-order methods
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2008 Prof. Gesine Reinert 1 Data x = x 1, x 2,..., x n, realisations of random variables X 1, X 2,..., X n with distribution (model)
More informationShunlong Luo. Academy of Mathematics and Systems Science Chinese Academy of Sciences
Superadditivity of Fisher Information: Classical vs. Quantum Shunlong Luo Academy of Mathematics and Systems Science Chinese Academy of Sciences luosl@amt.ac.cn Information Geometry and its Applications
More informationInformation Geometry on Hierarchy of Probability Distributions
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 47, NO. 5, JULY 2001 1701 Information Geometry on Hierarchy of Probability Distributions Shun-ichi Amari, Fellow, IEEE Abstract An exponential family or mixture
More informationNotes, March 4, 2013, R. Dudley Maximum likelihood estimation: actual or supposed
18.466 Notes, March 4, 2013, R. Dudley Maximum likelihood estimation: actual or supposed 1. MLEs in exponential families Let f(x,θ) for x X and θ Θ be a likelihood function, that is, for present purposes,
More information4 Hilbert spaces. The proof of the Hilbert basis theorem is not mathematics, it is theology. Camille Jordan
The proof of the Hilbert basis theorem is not mathematics, it is theology. Camille Jordan Wir müssen wissen, wir werden wissen. David Hilbert We now continue to study a special class of Banach spaces,
More informationStatistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach
Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Observed likelihood 3 Mean Score
More informationHypothesis Testing. 1 Definitions of test statistics. CB: chapter 8; section 10.3
Hypothesis Testing CB: chapter 8; section 0.3 Hypothesis: statement about an unknown population parameter Examples: The average age of males in Sweden is 7. (statement about population mean) The lowest
More informationINDIAN INSTITUTE OF TECHNOLOGY BOMBAY MA205 Complex Analysis Autumn 2012
INDIAN INSTITUTE OF TECHNOLOGY BOMBAY MA205 Complex Analysis Autumn 2012 September 5, 2012 Mapping Properties Lecture 13 We shall once again return to the study of general behaviour of holomorphic functions
More informationSpectral Theory of Orthogonal Polynomials
Spectral Theory of Orthogonal Barry Simon IBM Professor of Mathematics and Theoretical Physics California Institute of Technology Pasadena, CA, U.S.A. Lecture 2: Szegö Theorem for OPUC for S Spectral Theory
More informationWarped Product Bi-Slant Submanifolds of Cosymplectic Manifolds
Filomat 31:16 (2017) 5065 5071 https://doi.org/10.2298/fil1716065a Published by Faculty of Sciences and Mathematics University of Niš Serbia Available at: http://www.pmf.ni.ac.rs/filomat Warped Product
More informationConvergence of generalized entropy minimizers in sequences of convex problems
Proceedings IEEE ISIT 206, Barcelona, Spain, 2609 263 Convergence of generalized entropy minimizers in sequences of convex problems Imre Csiszár A Rényi Institute of Mathematics Hungarian Academy of Sciences
More informationProperly forking formulas in Urysohn spaces
Properly forking formulas in Urysohn spaces Gabriel Conant University of Notre Dame gconant@nd.edu March 4, 2016 Abstract In this informal note, we demonstrate the existence of forking and nondividing
More informationChi-square lower bounds
IMS Collections Borrowing Strength: Theory Powering Applications A Festschrift for Lawrence D. Brown Vol. 6 (2010) 22 31 c Institute of Mathematical Statistics, 2010 DOI: 10.1214/10-IMSCOLL602 Chi-square
More informationPHYS 352 Homework 2 Solutions
PHYS 352 Homework 2 Solutions Aaron Mowitz (, 2, and 3) and Nachi Stern (4 and 5) Problem The purpose of doing a Legendre transform is to change a function of one or more variables into a function of variables
More informationOn the Ladyzhenskaya Smagorinsky turbulence model of the Navier Stokes equations in smooth domains. The regularity problem
J. Eur. Math. Soc. 11, 127 167 c European Mathematical Society 2009 H. Beirão da Veiga On the Ladyzhenskaya Smagorinsky turbulence model of the Navier Stokes equations in smooth domains. The regularity
More informationAn exponential family of distributions is a parametric statistical model having densities with respect to some positive measure λ of the form.
Stat 8112 Lecture Notes Asymptotics of Exponential Families Charles J. Geyer January 23, 2013 1 Exponential Families An exponential family of distributions is a parametric statistical model having densities
More informationECE 275B Homework # 1 Solutions Version Winter 2015
ECE 275B Homework # 1 Solutions Version Winter 2015 1. (a) Because x i are assumed to be independent realizations of a continuous random variable, it is almost surely (a.s.) 1 the case that x 1 < x 2
More informationST5215: Advanced Statistical Theory
Department of Statistics & Applied Probability Wednesday, October 19, 2011 Lecture 17: UMVUE and the first method of derivation Estimable parameters Let ϑ be a parameter in the family P. If there exists
More informationThermostatistics based on Kolmogorov Nagumo averages: unifying framework for extensive and nonextensive generalizations
7 June 2002 Physics Letters A 298 (2002 369 374 wwwelseviercom/locate/pla Thermostatistics based on Kolmogorov Nagumo averages: unifying framewor for extensive and nonextensive generalizations Mare Czachor
More informationOn the Chi square and higher-order Chi distances for approximating f-divergences
c 2013 Frank Nielsen, Sony Computer Science Laboratories, Inc. 1/17 On the Chi square and higher-order Chi distances for approximating f-divergences Frank Nielsen 1 Richard Nock 2 www.informationgeometry.org
More informationEntropy-dissipation methods I: Fokker-Planck equations
1 Entropy-dissipation methods I: Fokker-Planck equations Ansgar Jüngel Vienna University of Technology, Austria www.jungel.at.vu Introduction Boltzmann equation Fokker-Planck equations Degenerate parabolic
More informationLecture 8 Plus properties, merit functions and gap functions. September 28, 2008
Lecture 8 Plus properties, merit functions and gap functions September 28, 2008 Outline Plus-properties and F-uniqueness Equation reformulations of VI/CPs Merit functions Gap merit functions FP-I book:
More informationInference. Data. Model. Variates
Data Inference Variates Model ˆθ (,..., ) mˆθn(d) m θ2 M m θ1 (,, ) (,,, ) (,, ) α = :=: (, ) F( ) = = {(, ),, } F( ) X( ) = Γ( ) = Σ = ( ) = ( ) ( ) = { = } :=: (U, ) , = { = } = { = } x 2 e i, e j
More information