On Information and Sufficiency
|
|
- Christian Stevenson
- 5 years ago
- Views:
Transcription
1 On Information and Sufficienc Huaiu hu SFI WORKING PAPER: SFI Working Papers contain accounts of scientific work of the author(s) and do not necessaril represent the views of the Santa Fe Institute. We accept papers intended for publication in peer-reviewed journals or proceedings volumes, but not papers that have alread appeared in print. Ecept for papers b our eternal facult, papers must be based on work done at SFI, inspired b an invited visit to or collaboration at SFI, or funded b an SFI grant. NOTICE: This working paper is included b permission of the contributing author(s) as a means to ensure timel distribution of the scholarl and technical work on a non-commercial basis. Copright and all rights therein are maintained b the author(s). It is understood that all persons coping this information will adhere to the terms and constraints invoked b each author's copright. These works ma be reposted onl with the eplicit permission of the copright holder. SANTA FE INSTITUTE
2 SFI working paper ftp://ftp.santafe.edu/pub/zhuh/suff-dev.ps. Submitted to Annals of Statistics on Jan 7, 997. Revised. On Information and Sucienc Huaiu hu March 3, 997 Abstract The information deviation between an two nite measures cannot be increased b an statistical operations (Markov morphisms). It is invariant if and onl if the morphism is sucient for these two measures. Abbreviated Title: Information and Sucienc Aliation: Santa Fe Institute, 399 Hde Park Road, Santa Fe, NM 8750, USA. zhuh@santafe.edu, Tel: , Fa: AMS 99 Classication: Primar 62B0 (stat info) Secondar 62B20 (meas th), 62B05 (su stat), 94A7 (meas of info). Kewords: information, sucienc. Acknowledgment: I thank Kim Sau Chung for pointing out several inaccuracies in a previous version. M work in SFI is sponsored b TXN, Inc. Introduction The -deviations, 2 [0 ], generalized to the space of nite measures, plas a uniquel important role in statistical inference (hu and Rohwer, 995). One basic requirement for a measure of information is that it should not be increased b an statistical operation, and should be invariant if and onl if the operation is sucient. The most general form of statistical operation is Markov morphism ( Cencov, 982), which includes deterministic mapping as a special case. It was proved (Kullback andleibler, 95) that the -deviation (known as the Kullback-Leibler divergence) for probabilit measures satises this criterion for deterministic mappings. Here we etend this result to all -deviations on an nite measures and for an Markov morphisms. A b-product is a concise proof of sucienc of MLE, slightl dierent from Fisher's original, which is also reproduced here.
3 2 Background We generall follow (Halmos and Savage, 949) for notation and terminolog concerning measures, with the eception that, following (hu, 996), the Radon-Nikodm densit will be denoted as quotient instead of derivative. If A is a proposition then A() [p] means A is satised b all ecept a set of measure zero in p. Let e P the space of nite measures. The -deviations between p q 2 e P is dened as D (p q) := 8 >< >: p +(; )q ; p q ; 2 (0 ) ( ; ) D (p q) =0 lim!0 lim! D (p q) =: We shall make use of the following simple properties (hu and Rohwer, 995). (2.) D (p q) 0: D (p q) =0 () p = q: (2.2) Let [X X ] and [Y Y] betwo measurable spaces. Denote b P e X and P e Y the corresponding spaces of nite measures, and b P X and P Y the spaces of probabilit measures. A Markov morphism ( Cencov, 982), also called a Markov kernel or a transitional probabilit distribution, is a mapping P j : X Y! R + (2.3) such that for each it is a probabilit measure on Y. It is also a mapping from e P X to e P Y which preserves total mass p = P j p p = p : (2.4) Let M = fp gp e X and N = fp gp e Y is sucient for N if there eists a Markov morphism P j such that N P j M (Cencov, 982). If, furthermore, P j M is also sucient for M, we sa that P j is sucient for M. This means that we have a Markov morphism P j which maps P j M back to M. For an given p and P j, there alwas eists an inverse kernel P j given b the Baes theorem, mapping p back top (Kolmogorov, 956). It ma be dierent for dierent p even if P j is ed. The condition of sucienc is that one ed inverse kernel P j works for the whole model M. Its intuitive meaning is as follows: After sampling from a particular p and transforming it to according to P j, it is alwas possible, using the inverse kernel P j, to generate from a \mock sample" of with eactl the same distribution p. Therefore keeping and discarding will not lose an information concerning the original distribution, as long as we do not need knowledge of p for the inverse kernel. See also (Fisher, 925, 4) and (Halmos and Savage, 949, 0). One important special case of Markov morphism is a deterministic mapping T : X! Y, as considered in (Fisher, 925 Halmos and Savage, 949 Kullback and Leibler, 95). The transformed variable = T () is called a statistic Its measure is p = p T ;. be two families of measures. We sa that M 2
4 3 Sucienc According to Information Deviations Consider a Markov morphism P j which transforms p q 2 e P X into p q 2 e P Y : p = P j p q = Our main result in this note is that P j q : (3.) D (p q ) D (p q ) (3.2) and that the equalit holdsif and onl if P j is sucient for fp q g. Denote the joint distributions as p and q, and the Baes inverse as P j and Q j, Then p := P j p = P j p q := P j q = Q j q : (3.3) p q; = P j p q; = p q ; = p q; P j Q; j : (3.4) For 2 (0 ), denoting r := p q;, we have D (p q ) ; D (p q )= ( ; ) = r ; P ( ; j ) Q; j = p q; ; p q; rd (P j Q j ): (3.5) B taking limits of we see this is true also for 2f0 g. Therefore, D (p q ) D (p q ), for 2 [0 ]. The equalit holds if and onl if P j = Q j [r]: (3.6) The eceptional set E with r(e) = 0 can be decomposed into E = E p [ E q such that p(e p )=q(e q )=0. B changing P j to Q j on E p,we have p = P j p q = P j q : (3.7) That is, P j is sucient for fp q g. The main thread of this proof is essentiall the same as that of (Kullback and Leibler, 95). This theorem establishes that the -deviations as measures of information are selfconsistent and keeps track of all the information in a statistical problem. 3
5 4 Sucienc According to the Information Metric For smooth nite dimensional models as commonl emploed in statistical problems, we can recover Fisher's proof (Fisher, 925, 7,0) of the sucienc and ecienc of maimum likelihood estimates, b dierentiating the formulas term b term in the above proof. Despite man later epositions, Fisher's original proof remains enigmatic to most statisticians. Therefore we consider it appropriate here to reproduce his proof in modern notations, which aords more generalit and rigor while being more concise and intuitive. Let M = fp g be a famil of measures parameterized b a vector = [ ::: m ]. The partial derivatives are i i. Let P j be a ed Markov morphism. Denote b l l l L j L j the log-likelihoods corresponding to p p p P j P j. We use hi and h i to denote the mean and variance, respectivel, with possible subscripts to denote conditioning. Then, b consecutivel taking logarithm, dierentiating, and taking average b P j, we obtain l = L j + l = L j + l : i l i l i L j i l : (4.2) h@ i li = h@ i l i i l : (4.3) Therefore, using the notation of (Amari, 985, Ch. 7) for the information metric, g ij () ; g ij () = i j l ; i j l D E = i l@ j l ;h@ i li h@ j li = h@i j li =: gij (j) 0: (4.4) The inequalit is in the sense of positive deniteness. The equalit holds if and onl i l i l i L j = 0 [p ]. That is, there eists one unique P j for the whole model M. In other words, P j is sucient. If, furthermore, asmptotic normalit is assumed, i l ;g ij ()( j ; b j ) (4.5) where b is the MLE. So g ij (j) =0if and onl if b is constant for each. In other words, b is a function of. Hence if a sucient estimate eists, then MLE is sucient. In an case it is alwas among the most ecient estimates. Fisher's original proof was in fact based on ;@ j l j p p 2 i@ j p p = h@ i li h@ j li ;h@ i l@ j l j li = g ij () ;h@ i j li (4.6) using g ij = ; R p@ j l instead of g ij = R p@ i l@ j l. The advantage of considerations based on information deviations instead of the information metric is, as noted in (Halmos and Savage, 949), that neither the proof nor its interpretation requires assumptions about smoothness, dimensionalit, dominatedness, asmptotic normalit, or an other similarl \etraneous" assumptions. 4
6 References Amari, S. (985). Dierential-Geometrical Methods in Statistics, volume 28 of Springer Lecture Notes in Statistics. Springer-Verlag, New York. Cencov, N. N. (982). Optimal Decision Rules and Optimal Inference. Amer. Math. Soc., Rhode Island. Translation from Russian, 972, Nauka, Moscow. Fisher, R. A. (925). Theor of statistical estimation. Proc. Camb. Phi. Soc., 22:700{725. Halmos, P. R. and Savage, L. J. (949). Application of the Radon-Nikodm theorem to the theor of sucient statistics. Ann. Math. Statist., pages 225{24. Kolmogorov, A. N. (956). Foundations of the Theor of Probabilit. Chelsea., New York. Translation of Grundbegrie der Wahrscheinlichkeitsrechnung, 933. Kullback, S. and Leibler, R. A. (95). On information and sucienc. Ann. Math. Statist., 22:79{86. hu, H. (996). On topologies and geometries of information deviations of nite measures. Submitted. hu, H. and Rohwer, R. (995). Baesian geometric theor of statistical inference. Submitted to Ann. Stat. 5
On the Speed of Quantum Computers with Finite Size Clocks
On the Speed of Quantum Computers with Finite Size Clocks Tino Gramss SFI WORKING PAPER: 1995-1-8 SFI Working Papers contain accounts of scientific work of the author(s) and do not necessarily represent
More informationCausal Effects for Prediction and Deliberative Decision Making of Embodied Systems
Causal Effects for Prediction and Deliberative Decision Making of Embodied ystems Nihat y Keyan Zahedi FI ORKING PPER: 2011-11-055 FI orking Papers contain accounts of scientific work of the author(s)
More informationROYAL INSTITUTE OF TECHNOLOGY KUNGL TEKNISKA HÖGSKOLAN. Department of Signals, Sensors & Systems
The Evil of Supereciency P. Stoica B. Ottersten To appear as a Fast Communication in Signal Processing IR-S3-SB-9633 ROYAL INSTITUTE OF TECHNOLOGY Department of Signals, Sensors & Systems Signal Processing
More informationThe Bootstrap is Inconsistent with Probability Theory
The Bootstrap is Inconsistent with Probability Theory David H. Wolpert SFI WORKING PAPER: 1995-10-091 SFI Working Papers contain accounts of scientific work of the author(s) and do not necessarily represent
More informationCharacterization of the Skew-Normal Distribution Via Order Statistics and Record Values
International Journal of Statistics and Probabilit; Vol. 4, No. 1; 2015 ISSN 1927-7032 E-ISSN 1927-7040 Published b Canadian Center of Science and Education Characterization of the Sew-Normal Distribution
More information(Anti-)Stable Points and the Dynamics of Extended Systems
(Anti-)Stable Points and the Dynamics of Extended Systems P.-M. Binder SFI WORKING PAPER: 1994-02-009 SFI Working Papers contain accounts of scientific work of the author(s) and do not necessarily represent
More informationOn the Stability of a Differential-Difference Analogue of a Two-Dimensional Problem of Integral Geometry
Filomat 3:3 18, 933 938 https://doi.org/1.98/fil183933b Published b Facult of Sciences and Mathematics, Universit of Niš, Serbia Available at: http://www.pmf.ni.ac.rs/filomat On the Stabilit of a Differential-Difference
More informationINF Introduction to classifiction Anne Solberg Based on Chapter 2 ( ) in Duda and Hart: Pattern Classification
INF 4300 151014 Introduction to classifiction Anne Solberg anne@ifiuiono Based on Chapter 1-6 in Duda and Hart: Pattern Classification 151014 INF 4300 1 Introduction to classification One of the most challenging
More informationAn Information Theory For Preferences
An Information Theor For Preferences Ali E. Abbas Department of Management Science and Engineering, Stanford Universit, Stanford, Ca, 94305 Abstract. Recent literature in the last Maimum Entrop workshop
More informationThe Entropy Power Inequality and Mrs. Gerber s Lemma for Groups of order 2 n
The Entrop Power Inequalit and Mrs. Gerber s Lemma for Groups of order 2 n Varun Jog EECS, UC Berkele Berkele, CA-94720 Email: varunjog@eecs.berkele.edu Venkat Anantharam EECS, UC Berkele Berkele, CA-94720
More information4.3 Exercises. local maximum or minimum. The second derivative is. e 1 x 2x 1. f x x 2 e 1 x 1 x 2 e 1 x 2x x 4
SECTION 4.3 HOW DERIVATIVES AFFECT THE SHAPE OF A GRAPH 297 local maimum or minimum. The second derivative is f 2 e 2 e 2 4 e 2 4 Since e and 4, we have f when and when 2 f. So the curve is concave downward
More informationEstimators in simple random sampling: Searls approach
Songklanakarin J Sci Technol 35 (6 749-760 Nov - Dec 013 http://wwwsjstpsuacth Original Article Estimators in simple random sampling: Searls approach Jirawan Jitthavech* and Vichit Lorchirachoonkul School
More informationProbability Theory, Random Processes and Mathematical Statistics
Probability Theory, Random Processes and Mathematical Statistics Mathematics and Its Applications Managing Editor: M.HAZEWINKEL Centre for Mathematics and Computer Science, Amsterdam, The Netherlands Volume
More informationMaximum Likelihood Estimation
Connexions module: m11446 1 Maximum Likelihood Estimation Clayton Scott Robert Nowak This work is produced by The Connexions Project and licensed under the Creative Commons Attribution License Abstract
More informationOn Irreversible Radiation Processes
On Irreversible Radiation Processes Huaiyu Zhu SFI WORKING PAPER: 1997-03-0 SFI Working Papers contain accounts of scientific work of the author(s) and do not necessarily represent the views of the Santa
More informationCONVEX FUNCTIONS WITH RESPECT TO A MEAN AND A CHARACTERIZATION OF QUASI-ARITHMETIC MEANS
Real Analsis Echange Vol. 29(1), 2003/2004, pp. 229246 Janusz Matkowski, Institute of Mathematics, Universit of Zielona Góra, PL-65-246 Zielona Góra, Poland. email: J.Matkowski@im.uz.zgora.pl CONVEX FUNCTIONS
More informationResearch Design - - Topic 15a Introduction to Multivariate Analyses 2009 R.C. Gardner, Ph.D.
Research Design - - Topic 15a Introduction to Multivariate Analses 009 R.C. Gardner, Ph.D. Major Characteristics of Multivariate Procedures Overview of Multivariate Techniques Bivariate Regression and
More informationMATRIX KERNELS FOR THE GAUSSIAN ORTHOGONAL AND SYMPLECTIC ENSEMBLES
Ann. Inst. Fourier, Grenoble 55, 6 (5), 197 7 MATRIX KERNELS FOR THE GAUSSIAN ORTHOGONAL AND SYMPLECTIC ENSEMBLES b Craig A. TRACY & Harold WIDOM I. Introduction. For a large class of finite N determinantal
More informationNearly Maximally Predictive Features and Their Dimensions
Nearl Maximall Predictive Features and Their Dimensions Sarah E. Marzen James P. Crutchfield SFI WORKING PAPER: 017-0-007 SFI Working Papers contain accounts of scienti5ic work of the author(s) and do
More informationMAT 1275: Introduction to Mathematical Analysis. Graphs and Simplest Equations for Basic Trigonometric Functions. y=sin( x) Function
MAT 275: Introduction to Mathematical Analsis Dr. A. Rozenblum Graphs and Simplest Equations for Basic Trigonometric Functions We consider here three basic functions: sine, cosine and tangent. For them,
More informationStiffness and Strength Tailoring in Uniform Space-Filling Truss Structures
NASA Technical Paper 3210 April 1992 Stiffness and Strength Tailoring in Uniform Space-Filling Truss Structures Mark S. Lake Summar This paper presents a deterministic procedure for tailoring the continuum
More informationAverage Reward Parameters
Simulation-Based Optimization of Markov Reward Processes: Implementation Issues Peter Marbach 2 John N. Tsitsiklis 3 Abstract We consider discrete time, nite state space Markov reward processes which depend
More informationSubadditivity of entropy for stochastic matrices
Subadditivity of entropy for stochastic matrices Wojciech S lomczyński 1 PREPRIT 2001/16 Abstract The entropy of irreducible stochastic matrix measures its mixing properties. We show that this quantity
More informationCopyright, 2008, R.E. Kass, E.N. Brown, and U. Eden REPRODUCTION OR CIRCULATION REQUIRES PERMISSION OF THE AUTHORS
Copright, 8, RE Kass, EN Brown, and U Eden REPRODUCTION OR CIRCULATION REQUIRES PERMISSION OF THE AUTHORS Chapter 6 Random Vectors and Multivariate Distributions 6 Random Vectors In Section?? we etended
More informationLogarithms. Bacteria like Staph aureus are very common.
UNIT 10 Eponentials and Logarithms Bacteria like Staph aureus are ver common. Copright 009, K1 Inc. All rights reserved. This material ma not be reproduced in whole or in part, including illustrations,
More information10 Back to planar nonlinear systems
10 Back to planar nonlinear sstems 10.1 Near the equilibria Recall that I started talking about the Lotka Volterra model as a motivation to stud sstems of two first order autonomous equations of the form
More informationQuantum Theory and Irreversibility
Quantum Theory and Irreversibility Huaiyu hu SFI WORKING PAPER: 1997-03-021 SFI Working Papers contain accounts of scientific work of the author(s) and do not necessarily represent the views of the Santa
More informationThe Sigmoidal Approximation of Łukasiewicz Operators
The Sigmoidal Approimation of Łukasiewicz Operators József Dombi, Zsolt Gera Universit of Szeged, Institute of Informatics e-mail: {dombi gera}@inf.u-szeged.hu Abstract: In this paper we propose an approimation
More informationOptimal scaling of the random walk Metropolis on elliptically symmetric unimodal targets
Optimal scaling of the random walk Metropolis on ellipticall smmetric unimodal targets Chris Sherlock 1 and Gareth Roberts 2 1. Department of Mathematics and Statistics, Lancaster Universit, Lancaster,
More informationSection 1.2: A Catalog of Functions
Section 1.: A Catalog of Functions As we discussed in the last section, in the sciences, we often tr to find an equation which models some given phenomenon in the real world - for eample, temperature as
More informationICES REPORT Model Misspecification and Plausibility
ICES REPORT 14-21 August 2014 Model Misspecification and Plausibility by Kathryn Farrell and J. Tinsley Odena The Institute for Computational Engineering and Sciences The University of Texas at Austin
More informationReview of Elementary Probability Lecture I Hamid R. Rabiee
Stochastic Processes Review o Elementar Probabilit Lecture I Hamid R. Rabiee Outline Histor/Philosoph Random Variables Densit/Distribution Functions Joint/Conditional Distributions Correlation Important
More informationProbability Theory Refresher
Machine Learning WS24 Module IN264 Sheet 2 Page Machine Learning Worksheet 2 Probabilit Theor Refresher Basic Probabilit Problem : A secret government agenc has developed a scanner which determines whether
More informationg(.) 1/ N 1/ N Decision Decision Device u u u u CP
Distributed Weak Signal Detection and Asymptotic Relative Eciency in Dependent Noise Hakan Delic Signal and Image Processing Laboratory (BUSI) Department of Electrical and Electronics Engineering Bogazici
More informationINF Anne Solberg One of the most challenging topics in image analysis is recognizing a specific object in an image.
INF 4300 700 Introduction to classifiction Anne Solberg anne@ifiuiono Based on Chapter -6 6inDuda and Hart: attern Classification 303 INF 4300 Introduction to classification One of the most challenging
More informationHamiltonicity and Fault Tolerance
Hamiltonicit and Fault Tolerance in the k-ar n-cube B Clifford R. Haithcock Portland State Universit Department of Mathematics and Statistics 006 In partial fulfillment of the requirements of the degree
More informationReceived: 20 December 2011; in revised form: 4 February 2012 / Accepted: 7 February 2012 / Published: 2 March 2012
Entropy 2012, 14, 480-490; doi:10.3390/e14030480 Article OPEN ACCESS entropy ISSN 1099-4300 www.mdpi.com/journal/entropy Interval Entropy and Informative Distance Fakhroddin Misagh 1, * and Gholamhossein
More informationComputation of Csiszár s Mutual Information of Order α
Computation of Csiszár s Mutual Information of Order Damianos Karakos, Sanjeev Khudanpur and Care E. Priebe Department of Electrical and Computer Engineering and Center for Language and Speech Processing
More informationJensen-Shannon Divergence and Hilbert space embedding
Jensen-Shannon Divergence and Hilbert space embedding Bent Fuglede and Flemming Topsøe University of Copenhagen, Department of Mathematics Consider the set M+ 1 (A) of probability distributions where A
More informationLaw of Cosines and Shannon-Pythagorean Theorem for Quantum Information
In Geometric Science of Information, 2013, Paris. Law of Cosines and Shannon-Pythagorean Theorem for Quantum Information Roman V. Belavkin 1 School of Engineering and Information Sciences Middlesex University,
More informationINTRODUCTION TO DIFFERENTIAL EQUATIONS
INTRODUCTION TO DIFFERENTIAL EQUATIONS. Definitions and Terminolog. Initial-Value Problems.3 Differential Equations as Mathematical Models CHAPTER IN REVIEW The words differential and equations certainl
More informationThe approximation of piecewise linear membership functions and Łukasiewicz operators
Fuzz Sets and Sstems 54 25 275 286 www.elsevier.com/locate/fss The approimation of piecewise linear membership functions and Łukasiewicz operators József Dombi, Zsolt Gera Universit of Szeged, Institute
More informationarxiv:math/ v1 [math.st] 19 Jan 2005
ON A DIFFERENCE OF JENSEN INEQUALITY AND ITS APPLICATIONS TO MEAN DIVERGENCE MEASURES INDER JEET TANEJA arxiv:math/05030v [math.st] 9 Jan 005 Let Abstract. In this paper we have considered a difference
More informationMATHEMATICAL ENGINEERING TECHNICAL REPORTS. A Superharmonic Prior for the Autoregressive Process of the Second Order
MATHEMATICAL ENGINEERING TECHNICAL REPORTS A Superharmonic Prior for the Autoregressive Process of the Second Order Fuyuhiko TANAKA and Fumiyasu KOMAKI METR 2006 30 May 2006 DEPARTMENT OF MATHEMATICAL
More informationINF Introduction to classifiction Anne Solberg
INF 4300 8.09.17 Introduction to classifiction Anne Solberg anne@ifi.uio.no Introduction to classification Based on handout from Pattern Recognition b Theodoridis, available after the lecture INF 4300
More informationCOMPACT DIFFERENCE OF WEIGHTED COMPOSITION OPERATORS ON N p -SPACES IN THE BALL
COMPACT DIFFERENCE OF WEIGHTED COMPOSITION OPERATORS ON N p -SPACES IN THE BALL HU BINGYANG and LE HAI KHOI Communicated by Mihai Putinar We obtain necessary and sucient conditions for the compactness
More informationParameterized Joint Densities with Gaussian and Gaussian Mixture Marginals
Parameterized Joint Densities with Gaussian and Gaussian Miture Marginals Feli Sawo, Dietrich Brunn, and Uwe D. Hanebeck Intelligent Sensor-Actuator-Sstems Laborator Institute of Computer Science and Engineering
More informationCONTINUOUS SPATIAL DATA ANALYSIS
CONTINUOUS SPATIAL DATA ANALSIS 1. Overview of Spatial Stochastic Processes The ke difference between continuous spatial data and point patterns is that there is now assumed to be a meaningful value, s
More information36-720: Latent Class Models
36-720: Latent Class Models Brian Junker October 17, 2007 Latent Class Models and Simpson s Paradox Latent Class Model for a Table of Counts An Intuitive E-M Algorithm for Fitting Latent Class Models Deriving
More information338 Jin Suk Pak and Yang Jae Shin 2. Preliminaries Let M be a( + )-dimensional almost contact metric manifold with an almost contact metric structure
Comm. Korean Math. Soc. 3(998), No. 2, pp. 337-343 A NOTE ON CONTACT CONFORMAL CURVATURE TENSOR Jin Suk Pak* and Yang Jae Shin Abstract. In this paper we show that every contact metric manifold with vanishing
More informationDecision tables and decision spaces
Abstract Decision tables and decision spaces Z. Pawlak 1 Abstract. In this paper an Euclidean space, called a decision space is associated with ever decision table. This can be viewed as a generalization
More informationFunctions and Graphs TERMINOLOGY
5 Functions and Graphs TERMINOLOGY Arc of a curve: Part or a section of a curve between two points Asmptote: A line towards which a curve approaches but never touches Cartesian coordinates: Named after
More informationChapter 1: A Brief Review of Maximum Likelihood, GMM, and Numerical Tools. Joan Llull. Microeconometrics IDEA PhD Program
Chapter 1: A Brief Review of Maximum Likelihood, GMM, and Numerical Tools Joan Llull Microeconometrics IDEA PhD Program Maximum Likelihood Chapter 1. A Brief Review of Maximum Likelihood, GMM, and Numerical
More informationA sharp Blaschke Santaló inequality for α-concave functions
Noname manuscript No will be inserted b the editor) A sharp Blaschke Santaló inequalit for α-concave functions Liran Rotem the date of receipt and acceptance should be inserted later Abstract We define
More informationMax. Likelihood Estimation. Outline. Econometrics II. Ricardo Mora. Notes. Notes
Maximum Likelihood Estimation Econometrics II Department of Economics Universidad Carlos III de Madrid Máster Universitario en Desarrollo y Crecimiento Económico Outline 1 3 4 General Approaches to Parameter
More information1.6 ELECTRONIC STRUCTURE OF THE HYDROGEN ATOM
1.6 ELECTRONIC STRUCTURE OF THE HYDROGEN ATOM 23 How does this wave-particle dualit require us to alter our thinking about the electron? In our everda lives, we re accustomed to a deterministic world.
More informationConsistency of the maximum likelihood estimator for general hidden Markov models
Consistency of the maximum likelihood estimator for general hidden Markov models Jimmy Olsson Centre for Mathematical Sciences Lund University Nordstat 2012 Umeå, Sweden Collaborators Hidden Markov models
More informationMinimal enumerations of subsets of a nite set and the middle level problem
Discrete Applied Mathematics 114 (2001) 109 114 Minimal enumerations of subsets of a nite set and the middle level problem A.A. Evdokimov, A.L. Perezhogin 1 Sobolev Institute of Mathematics, Novosibirsk
More informationA PARAMETRIC MODEL FOR DISCRETE-VALUED TIME SERIES. 1. Introduction
tm Tatra Mt. Math. Publ. 00 (XXXX), 1 10 A PARAMETRIC MODEL FOR DISCRETE-VALUED TIME SERIES Martin Janžura and Lucie Fialová ABSTRACT. A parametric model for statistical analysis of Markov chains type
More informationStochastic Interpretation for the Arimoto Algorithm
Stochastic Interpretation for the Arimoto Algorithm Serge Tridenski EE - Sstems Department Tel Aviv Universit Tel Aviv, Israel Email: sergetr@post.tau.ac.il Ram Zamir EE - Sstems Department Tel Aviv Universit
More informationSome New Information Inequalities Involving f-divergences
BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 12, No 2 Sofia 2012 Some New Information Inequalities Involving f-divergences Amit Srivastava Department of Mathematics, Jaypee
More informationAn analytic proof of the theorems of Pappus and Desargues
Note di Matematica 22, n. 1, 2003, 99 106. An analtic proof of the theorems of Pappus and Desargues Erwin Kleinfeld and Tuong Ton-That Department of Mathematics, The Universit of Iowa, Iowa Cit, IA 52242,
More informationSupplementary Material: A Robust Approach to Sequential Information Theoretic Planning
Supplementar aterial: A Robust Approach to Sequential Information Theoretic Planning Sue Zheng Jason Pacheco John W. Fisher, III. Proofs of Estimator Properties.. Proof of Prop. Here we show how the bias
More informationThe uniformly accelerated motion in General Relativity from a geometric point of view. 1. Introduction. Daniel de la Fuente
XI Encuentro Andaluz de Geometría IMUS (Universidad de Sevilla), 15 de mayo de 2015, págs. 2934 The uniformly accelerated motion in General Relativity from a geometric point of view Daniel de la Fuente
More informationFuzzy Topology On Fuzzy Sets: Regularity and Separation Axioms
wwwaasrcorg/aasrj American Academic & Scholarl Research Journal Vol 4, No 2, March 212 Fuzz Topolog n Fuzz Sets: Regularit and Separation Aioms AKandil 1, S Saleh 2 and MM Yakout 3 1 Mathematics Department,
More informationarxiv: v2 [cs.it] 26 Sep 2011
Sequences o Inequalities Among New Divergence Measures arxiv:1010.041v [cs.it] 6 Sep 011 Inder Jeet Taneja Departamento de Matemática Universidade Federal de Santa Catarina 88.040-900 Florianópolis SC
More informationAnalysis of the AIC Statistic for Optimal Detection of Small Changes in Dynamic Systems
Analysis of the AIC Statistic for Optimal Detection of Small Changes in Dynamic Systems Jeremy S. Conner and Dale E. Seborg Department of Chemical Engineering University of California, Santa Barbara, CA
More informationRenormalization Group Analysis of the Small-World Network Model
Renormalization Group Analysis of the Small-World Network Model M. E. J. Newman D. J. Watts SFI WORKING PAPER: 1999-04-029 SFI Working Papers contain accounts of scientific work of the author(s) and do
More informationPerturbation Theory for Variational Inference
Perturbation heor for Variational Inference Manfred Opper U Berlin Marco Fraccaro echnical Universit of Denmark Ulrich Paquet Apple Ale Susemihl U Berlin Ole Winther echnical Universit of Denmark Abstract
More information1. d = 1. or Use Only in Pilot Program F Review Exercises 131
or Use Onl in Pilot Program F 0 0 Review Eercises. Limit proof Suppose f is defined for all values of near a, ecept possibl at a. Assume for an integer N 7 0, there is another integer M 7 0 such that f
More informationL 2 Geometry of the Symplectomorphism Group
University of Notre Dame Workshop on Innite Dimensional Geometry, Vienna 2015 Outline 1 The Exponential Map on D s ω(m) 2 Existence of Multiplicity of Outline 1 The Exponential Map on D s ω(m) 2 Existence
More informationWeek #7 Maxima and Minima, Concavity, Applications Section 4.2
Week #7 Maima and Minima, Concavit, Applications Section 4.2 From Calculus, Single Variable b Hughes-Hallett, Gleason, McCallum et. al. Copright 2005 b John Wile & Sons, Inc. This material is used b permission
More informationEstimation of Mixed Exponentiated Weibull Parameters in Life Testing
International Mathematical Forum, Vol., 6, no. 6, 769-78 HIKARI Ltd, www.m-hikari.com http://d.doi.org/.988/imf.6.6445 Estimation of Mied Eponentiated Weibull Parameters in Life Testing A. M. Abd-Elrahman
More informationSurface Integrals (6A)
Surface Integrals (6A) Surface Integral Stokes' Theorem Copright (c) 2012 Young W. Lim. Permission is granted to cop, distribute and/or modif this document under the terms of the GNU Free Documentation
More informationFlows and Connectivity
Chapter 4 Flows and Connectivit 4. Network Flows Capacit Networks A network N is a connected weighted loopless graph (G,w) with two specified vertices and, called the source and the sink, respectivel,
More informationRearrangements and polar factorisation of countably degenerate functions G.R. Burton, School of Mathematical Sciences, University of Bath, Claverton D
Rearrangements and polar factorisation of countably degenerate functions G.R. Burton, School of Mathematical Sciences, University of Bath, Claverton Down, Bath BA2 7AY, U.K. R.J. Douglas, Isaac Newton
More informationLimits and Continuous Functions. 2.2 Introduction to Limits. We first interpret limits loosely. We write. lim f(x) = L
2 Limits and Continuous Functions 2.2 Introduction to Limits We first interpret limits loosel. We write lim f() = L and sa the limit of f() as approaches c, equals L if we can make the values of f() arbitraril
More informationz-axis SUBMITTED BY: Ms. Harjeet Kaur Associate Professor Department of Mathematics PGGCG 11, Chandigarh y-axis x-axis
z-ais - - SUBMITTED BY: - -ais - - - - - - -ais Ms. Harjeet Kaur Associate Proessor Department o Mathematics PGGCG Chandigarh CONTENTS: Function o two variables: Deinition Domain Geometrical illustration
More informationNested Inequalities Among Divergence Measures
Appl Math Inf Sci 7, No, 49-7 0 49 Applied Mathematics & Information Sciences An International Journal c 0 NSP Natural Sciences Publishing Cor Nested Inequalities Among Divergence Measures Inder J Taneja
More information8. BOOLEAN ALGEBRAS x x
8. BOOLEAN ALGEBRAS 8.1. Definition of a Boolean Algebra There are man sstems of interest to computing scientists that have a common underling structure. It makes sense to describe such a mathematical
More informationThe Pacic Institute for the Mathematical Sciences http://www.pims.math.ca pims@pims.math.ca Surprise Maximization D. Borwein Department of Mathematics University of Western Ontario London, Ontario, Canada
More information460 HOLGER DETTE AND WILLIAM J STUDDEN order to examine how a given design behaves in the model g` with respect to the D-optimality criterion one uses
Statistica Sinica 5(1995), 459-473 OPTIMAL DESIGNS FOR POLYNOMIAL REGRESSION WHEN THE DEGREE IS NOT KNOWN Holger Dette and William J Studden Technische Universitat Dresden and Purdue University Abstract:
More informationLinear programming: Theory
Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analsis and Economic Theor Winter 2018 Topic 28: Linear programming: Theor 28.1 The saddlepoint theorem for linear programming The
More informationTransversals in loops. 1. Elementary properties. 1. Introduction
Quasigroups and Related Systems 18 (2010), 43 58 Transversals in loops. 1. Elementary properties Eugene Kuznetsov Devoted to the memory of Valentin D. Belousov (1925-1988) Abstract. A new notion of a transversal
More information10.5 Graphs of the Trigonometric Functions
790 Foundations of Trigonometr 0.5 Graphs of the Trigonometric Functions In this section, we return to our discussion of the circular (trigonometric functions as functions of real numbers and pick up where
More informationOn the Monge-Kantorovich Duality Theorem. California State University, Sacramento and Universitat Freiburg
On the Monge-Kantorovich Duality Theorem By D. Ramachandran and 1 L. Ruschendorf California State University, Sacramento and Universitat Freiburg The Monge-Kantorovich duality theorem has a variety of
More informationThe exact region of stability for MacCormack scheme. Hoon Hong. Research Institute for Symbolic Computation. Johannes Kepler University
The eact region of stailit for MacCormack scheme Hoon Hong Research Institute for Smolic Computation Johannes Kepler Universit A- Linz, Austria e-mail: hhong@risc.uni-linz.ac.at Astract Let the two dimensional
More informationMt. Douglas Secondary
Foundations of Math 11 Section 7.1 Quadratic Functions 31 7.1 Quadratic Functions Mt. Douglas Secondar Quadratic functions are found in everda situations, not just in our math classroom. Tossing a ball
More information3 Undirected Graphical Models
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.438 Algorithms For Inference Fall 2014 3 Undirected Graphical Models In this lecture, we discuss undirected
More information4 Inverse function theorem
Tel Aviv Universit, 2013/14 Analsis-III,IV 53 4 Inverse function theorem 4a What is the problem................ 53 4b Simple observations before the theorem..... 54 4c The theorem.....................
More informationON THE BANACH-ISOMORPHIC CLASSIFICATION OF L p SPACES OF HYPERFINITE SEMIFINITE VON NEUMANN ALGEBRAS F. A. SUKOCHEV Abstract. We present a survey of r
ON THE BANACH-ISOMORPHIC CLASSIFICATION OF L p SPACES OF HYPERFINITE SEMIFINITE VON NEUMANN ALGEBRAS F. A. SUKOCHEV Abstract. We present a survey of recent results in the Banach space classication of non-commutative
More information2.5 CONTINUITY. a x. Notice that Definition l implicitly requires three things if f is continuous at a:
SECTION.5 CONTINUITY 9.5 CONTINUITY We noticed in Section.3 that the it of a function as approaches a can often be found simpl b calculating the value of the function at a. Functions with this propert
More informationAdditional Material On Recursive Sequences
Penn State Altoona MATH 141 Additional Material On Recursive Sequences 1. Graphical Analsis Cobweb Diagrams Consider a generic recursive sequence { an+1 = f(a n ), n = 1,, 3,..., = Given initial value.
More informationA PARTIAL CHARACTERIZATION OF THE COCIRCUITS OF A SPLITTING MATROID
DEPARTMENT OF MATHEMATICS TECHNICAL REPORT A PARTIAL CHARACTERIZATION OF THE COCIRCUITS OF A SPLITTING MATROID DR. ALLAN D. MILLS MAY 2004 No. 2004-2 TENNESSEE TECHNOLOGICAL UNIVERSITY Cookeville, TN 38505
More informationCarnegie Mellon University Forbes Ave. Pittsburgh, PA 15213, USA. fmunos, leemon, V (x)ln + max. cost functional [3].
Gradient Descent Approaches to Neural-Net-Based Solutions of the Hamilton-Jacobi-Bellman Equation Remi Munos, Leemon C. Baird and Andrew W. Moore Robotics Institute and Computer Science Department, Carnegie
More informationEntropy, Relative Entropy and Mutual Information Exercises
Entrop, Relative Entrop and Mutual Information Exercises Exercise 2.: Coin Flips. A fair coin is flipped until the first head occurs. Let X denote the number of flips required. (a) Find the entrop H(X)
More informationThe geometrical structure of quantum theory as a natural generalization of information geometry
The geometrical structure of quantum theory as a natural generalization of information geometry Marcel Reginatto Physikalisch-Technische Bundesanstalt, Braunschweig, Germany Abstract. Quantum mechanics
More informationMutual Information Approximation via. Maximum Likelihood Estimation of Density Ratio:
Mutual Information Approimation via Maimum Likelihood Estimation of Densit Ratio Taiji Suzuki Dept. of Mathematical Informatics, The Universit of Toko Email: s-taiji@stat.t.u-toko.ac.jp Masashi Sugiama
More informationUNIVERSIDAD CARLOS III DE MADRID MATHEMATICS II EXERCISES (SOLUTIONS )
UNIVERSIDAD CARLOS III DE MADRID MATHEMATICS II EXERCISES (SOLUTIONS ) CHAPTER : Limits and continuit of functions in R n. -. Sketch the following subsets of R. Sketch their boundar and the interior. Stud
More informationSome Polyomino Tilings of the Plane
Some Polyomino Tilings of the Plane Cristopher Moore SFI WORKING PAPER: 1999-04-03 SFI Working Papers contain accounts of scientific work of the author(s) and do not necessarily represent the views of
More information