Nonparametric estimation of log-concave densities
|
|
- Deirdre Wilcox
- 5 years ago
- Views:
Transcription
1 Nonparametric estimation of log-concave densities Jon A. Wellner University of Washington, Seattle Northwestern University November 5, 2010
2 Conference on Shape Restrictions in Non- and Semi-Parametric Estimation of Econometric Models Based on joint work with: Fadoua Balabdaoui Kaspar Rufibach Arseni Seregin
3 Outline A: Log-concave densities on R 1 B: C: D: E: F: Nonparametric estimation, log-concave on R Limit theory at a fixed point in R Estimation of the mode, log-concave density on R Generalizations: s concave densities on R and R d Summary; problems and open questions Northwestern University, November 5,
4 Suppose that A. Log-concave densities on R 1 f(x) f ϕ (x) = exp(ϕ(x)) = exp ( ( ϕ(x))) where ϕ is concave (and ϕ is convex). The class of all densities f on R of this form is called the class of log-concave densities, P log concave P 0. Properties of log-concave densities: A density f on R is log-concave if and only if its convolution with any unimodal density is again unimodal (Ibragimov, 1956). Every log-concave density f is unimodal (but need not be symmetric). P 0 is closed under convolution. Northwestern University, November 5,
5 A. Log-concave densities on R 1 Many parametric families are log-concave, for example: Normal (µ, σ 2 ) Uniform(a, b) Gamma(r, λ) for r 1 Beta(a, b) for a, b 1 t r densities with r > 0 are not log-concave Tails of log-concave densities are necessarily sub-exponential P log concave = the class of Polyá frequency functions of order 2, P F F 2, in the terminology of Schoenberg (1951) and Karlin (1968). See Marshall and Olkin (1979), chapter 18, and Dharmadhikari and Joag-Dev (1988), page 150. for nice introductions. Northwestern University, November 5,
6 B. Nonparametric estimation, log-concave on R The (nonparametric) MLE f n exists (Rufibach, Dümbgen and Rufibach). f n can be computed: and Rufibach) R-package logcondens (Dümbgen In contrast, the (nonparametric) MLE for the class of unimodal densities on R 1 does not exist. Birgé (1997) and Bickel and Fan (1996) consider alternatives to maximum likelihood for the class of unimodal densities. Consistency and rates of convergence for f n : Dümbgen and Rufibach, (2009); Pal, Woodroofe and Meyer (2007). Pointwise limit theory? Yes! Balabdaoui, Rufibach, and W (2009). Northwestern University, November 5,
7 B. Nonparametric estimation, log-concave on R MLE of f and ϕ: Let C denote the class of all concave function ϕ : R [, ). The estimator ϕ n based on X 1,..., X n i.i.d. as f 0 is the maximizer of the adjusted criterion function over ϕ C. l n (ϕ) = = logf ϕ (x)df n (x) ϕ(x)df n (x) e ϕ(x) dx f ϕ (x)dx Properties of f n, ϕ n : (Dümbgen & Rufibach, 2009) ϕ n is piecewise linear. ϕ n = on R \ [X (1), X (n) ]. The knots (or kinks) of ϕ n occur at a subset of the order statistics X (1) < X (2) < < X (n). Characterized by... Northwestern University, November 5,
8 B. Nonparametric estimation, log-concave on R... ϕ n is the MLE of logf 0 = ϕ 0 if and only if { Hn (x), for all x > X Ĥ n (x) (1), = H n (x), if x is a knot. where F n (x) = x X (1) f n (y)dy, Ĥn(x) = x x X (1) F n (y)dy, H n (x) = F n(y)dy. Furthermore, for every function such that ϕ n + t is concave for t small enough, (x)df n(x) (x)d F n (x). R R Northwestern University, November 5,
9 B. Nonparametric estimation, log-concave on R Consistency of f n and ϕ n : (Pal, Woodroofe, & Meyer, 2007): If f 0 P 0, then H( f n, f 0 ) a.s. 0. (Dümbgen & Rufibach, 2009): If f 0 P 0 and ϕ 0 H β,l (T ) for some compact T = [A, B] {x : f 0 (x) > 0}, M <, and 1 β 2. Then sup( ϕ n (t) ϕ 0 (t)) = O p t T sup t T n (ϕ 0 (t) ϕ n (t)) = O p ( logn n ( logn n ) β/(2β+1), ) β/(2β+1) and where T n [A + (logn/n) β/(2β+1), B (logn/n) β/(2β+1) ] and β/(2β + 1) [1/3, 2/5] for 1 β 2. The same remains true if ϕ n, ϕ 0 are replaced by f n, f 0. Northwestern University, November 5,
10 B. Nonparametric estimation, log-concave on R If ϕ 0 H β,l (T ) as above and, with ϕ 0 = ϕ 0( ) or ϕ 0 ( +), ϕ 0 (x) ϕ 0 (y) C(y x) for some C > 0 and all A x < y B, then sup t T n F n (t) F n (t) = O p ( logn n ) 3β/(4β+2) where 3β/(2β + 4) [1/2, 3/5] = [.5,.6] for 1 β 2.. If β > 1, this implies sup t Tn F n (t) F n (t) = o p (n 1/2 ). Northwestern University, November 5,
11 B. Nonparametric estimation, log-concave on R density functions density functions log density functions log density functions CDFs CDFs hazard functions hazard functions Northwestern University, November 5,
12 B. Nonparametric estimation, log-concave on R Northwestern University, November 5,
13 B. Nonparametric estimation, log-concave on R Northwestern University, November 5,
14 B. Nonparametric estimation, log-concave on R Northwestern University, November 5,
15 C: Limit theory at a fixed point in R Assumptions: f 0 is log-concave, f 0 (x 0 ) > 0. If ϕ 0 (x 0) 0, then k = 2; otherwise, k is the smallest integer such that ϕ (j) 0 (x 0) = 0, j = 2,..., k 1, ϕ (k) 0 (x 0) 0. ϕ (k) 0 is continuous in a neighborhood of x 0. Example: f 0 (x) = Cexp( x 4 ) with C = 2Γ(3/4)/π: k = 4. Driving process: Y k (t) = t 0 W (s)ds t k+2, W standard 2-sided Brownian motion. Invelope process: H k determined by limit Fenchel relations: H k (t) Y k (t) for all t R R (H k(t) Y k (t))dh (3) k (t) = 0. H (2) k is concave. Northwestern University, November 5,
16 C: Limit theory at a fixed point in R Theorem. (Balabdaoui, Rufibach, & W, 2009) Pointwise limit theorem for f n (x 0 ): ( n k/(2k+1) ( f n (x 0 ) f 0 (x 0 )) n (k 1)/(2k+1) ( f n(x 0 ) f 0 (x 0)) where c k d k ) d f 0(x 0 ) k+1 ϕ (k) 0 (x 0) (k + 2)! c kh (2) k (0) d k H (3) k (0) 1/(2k+1) f 0(x 0 ) k+2 ϕ (k) 0 (x 0) 3 [(k + 2)!] 3, 1/(2k+1). Northwestern University, November 5,
17 C: Limit theory at a fixed point in R Pointwise limit theorem for ϕ n (x 0 ): ( where n k/(2k+1) ( ϕ n (x 0 ) ϕ 0 (x 0 )) n (k 1)/(2k+1) ( ϕ n(x 0 ) ϕ 0 (x 0)) C k D k ϕ (k) 0 (x 0) f 0 (x 0 ) k (k + 2)! ) d 1/(2k+1) ϕ (k) 0 (x 0) 3 f 0 (x 0 ) k 1 [(k + 2)!] 3 C kh (2) k (0) D k H (3) k (0), 1/(2k+1). Proof: Use the same perturbation as for convex - decreasing density proof with perturbation version of characterization: Northwestern University, November 5,
18 C: Limit theory at a fixed point in R Τ Τ Northwestern University, November 5,
19 D: Mode estimation, log-concave density on R Let x 0 = M(f 0 ) be the mode of the log-concave density f 0, recalling that P 0 P unimodal. Lower bound calculations using Jongbloed s perturbation ϕ ɛ of ϕ 0 yields: Proposition. If f 0 P 0 satisfies f 0 (x 0 ) > 0, f 0 (x 0) < 0, and f 0 is continuous in a neighborhood of x 0, and T n is any estimator of the mode x 0 M(f 0 ), then f n exp(ϕ ɛn ) with ɛ n νn 1/5 and ν 2f 0 (x 0) 2 /(5f 0 (x 0 )), lim inf n n1/5 inf max {E n T n M(f n ), E 0 T n M(f 0 ) } T n ( ) 1/5 ( 5/2 f0 (x 0 ) ) 1/ e f 0 (x 0) 2 Does the MLE M( f n ) achieve this? Northwestern University, November 5,
20 D: Mode estimation, log-concave density on R Northwestern University, November 5,
21 D: Mode estimation, log-concave density on R Proposition. (Balabdaoui, Rufibach, & W, 2009) Suppose that f 0 P 0 satisfies: ϕ (j) 0 (x 0) = 0, j = 2,..., k 1, ϕ (k) 0 (x 0) 0, and ϕ (k) 0 is continuous in a neighborhood of x 0. Then M n M( f n ) min{u : f n (u) = sup t f n (t)}, satisfies n 1/(2k+1) ( M n M(f 0 )) d where M(H (2) k ) = argmax(h (2) k ). ((k + 2)!)2 f 0 (x 0 ) f (k) 0 (x 0) 2 1/(2k+1) M(H (2) k ) Note that when k = 2 this agrees with the lower bound calculation, at least up to absolute constants. Northwestern University, November 5,
22 D: Mode estimation, log-concave density on R f 2 8 f f Northwestern University, November 5,
23 D: Mode estimation, log-concave density on R When f 0 = φ, the standard normal density, M(f 0 ) = 0, f 0 (0) = (2π) 1/2, f 0 (0) = (2π) 1/2, and hence ((4)!)2 f 0 (0) f (2) 0 (x 0) 2 1/5 = ( 24 2 (2π) 1/2 (2π) 1 )1/5 = Northwestern University, November 5,
24 E: Generalizations of log-concave to R and R d : Three generalizations: log concave densities on R d (Cule, Samworth, and Stewart, 2010) s concave and h transformed convex densities on R d (Seregin, 2010) Hyperbolically k monotone and completely monotone densities on R; (Bondesson, 1981, 1992) Northwestern University, November 5,
25 E: Generalizations of log-concave to R and R d : Log-concave densities on R d : A density f on R d is log-concave if f(x) = exp(ϕ(x)) with ϕ concave. Some properties: Any log concave f is unimodal The level sets of f are closed convex sets Convolutions of log-concave distributions are log-concave. Marginals of log-concave distributions are log-concave. Northwestern University, November 5,
26 E: Generalizations of log-concave to R and R d : MLE of f P 0 (R d ): (Cule, Samworth, Stewart, 2010) MLE f n = argmax f P0 (R d ) P nlogf exists and is unique if n d + 1. The estimator ϕ n of ϕ 0 is a taut tent stretched over tent poles of certain heights at a subset of the observations. Computable via non-differentiable convex optimization methods: Shor s (1985) r algorithm: R package LogConcDEAD (Cule, Samworth, Stewart, 2008). Northwestern University, November 5,
27 E: Generalizations of log-concave to R and R d : Northwestern University, November 5,
28 E: Generalizations of log-concave to R and R d : If f 0 is any density on R d with R d x f 0 (x)dx <, R d f 0 (x)logf 0 (x)dx <, and {x R d : f 0 (x) > 0} = int(supp(f 0 )), then f n satisfies: f n (x) f (x) dx a.s. 0 R d where, for the Kullback-Leibler divergence K(f 0, f) = f 0 log(f 0 /f)dµ, f = argmin f P0 (R d ) K(f 0, f) is the pseudo-true density in P 0 (R d ) corresponding to f 0. In fact: R d ea x f n (x) f (x) dx a.s. 0 for any a < a 0 where f (x) exp( a 0 x + b 0 ). Northwestern University, November 5,
29 E: Generalizations of log-concave to R and R d : r concave and h transformed convex densities on R d : (Seregin, 2010; Seregin &, 2010) Generalization to s concave densities: A density f on R d is r concave on C R d if f(λx + (1 λ)y) M r (f(x), f(y); λ) for all x, y C and 0 < λ < 1 where M r (a, b; λ) = ((1 λ)a r + λb r ) 1/r, r 0, a, b > 0, 0, r < 0, ab = 0 a 1 λ b λ, r = 0. Let P r denote the class of all r concave densities on C. For r 0 it suffices to consider C = R d, and it is almost immediate from the definitions that if f P r for some r 0, then { g(x) f(x) = 1/r }, r < 0 for g convex. exp( g(x)), r = 0 Northwestern University, November 5,
30 E: Generalizations of log-concave to R and R d : Long history: Avriel (1972), Prékopa (1973), Borell (1975), Rinott (1976), Brascamp and Lieb (1976) Nice connections to t concave measures: (Borell, 1975) Known now in math-analysis as the Borell, Brascamp, Lieb inequality One way to get heavier tails than log-concave! Example: Multivariate t density with p degrees of freedom: if f(x) = f(x; p, d) = Γ((d + p)/2) 1 Γ(p/2)(pπ) d/2 ( 1 + x 2 p ) (d+p)/2 then f P 1/s for s (d, d + p]; i.e. f P r (R d ) for 1/(d + p) r < 1/d. Northwestern University, November 5,
31 E: Generalizations of log-concave to R and R d : A measure µ on (R, B) is called t concave if for all A, B B and 0 λ 1 µ(λa + (1 λ)b) M t (µ(a), µ(b), λ). Theorem. (Borell, 1975) If f P r with 1/d r, then the measure P = P f defined by P (A) = A f(x)dx for Borel subsets A of R d is t concave with and conversely. t = r 1+dr, if 1/d < r <,, if r = 1/d, 1/d, if r =, Northwestern University, November 5,
32 E: Generalizations of log-concave to R and R d : h convex densities: Seregin (2010), Seregin & W (2010)) f(x) = h(ϕ(x)) (1) where ϕ : R d R is convex, h : R R + is decreasing and continuous; e.g. h s (u) (1 + u/s) s with s > d. This motivates the following definition: Definition. Say that h : R R + is a decreasing transformation if, with y 0 sup{y : h(y) > 0}, y inf{y : h(y) < }, h(y) = o(y α ) for some α > d as y. If y >, then h(y) (y y ) β y y. for some β > d as If y =, then h(y) γ h( Cy) = o(1) as y for some γ, C > 0. h is continuously differentiable on (y, y 0 ). Northwestern University, November 5,
33 E: Generalizations of log-concave to R and R d : Northwestern University, November 5,
34 E: Generalizations of log-concave to R and R d : Let P h denote the collection of all densities on R d of the form f = h ϕ for a fixed decreasing transformation h and ϕ convex, and let f n argmax f Ph P n logf, the MLE. Theorem. f n P h exists if n n d where n d d + dγ1{y = } + βd2 α(β d) 1{y > } { d + 1, if h(y) = e y, = ), if h(y) = y s, s > d. d ( s s d Theorem. If h is a decreasing transformation as defined above, and f 0 P h, then H( f n, f 0 ) a.s. 0. Northwestern University, November 5,
35 E: Generalizations of log-concave to R and R d : Questions: Rates of convergence? MLE (rate-) inefficient for d 4? efficient rates? How to penalize to get Multivariate classes with nice preservation/closure properties and smoother than log-concave? Can we treat f n P h with miss-specification: f 0 / P h? Algorithms for computing f n P h? Northwestern University, November 5,
Nonparametric estimation under Shape Restrictions
Nonparametric estimation under Shape Restrictions Jon A. Wellner University of Washington, Seattle Statistical Seminar, Frejus, France August 30 - September 3, 2010 Outline: Five Lectures on Shape Restrictions
More informationNonparametric estimation of log-concave densities
Nonparametric estimation of log-concave densities Jon A. Wellner University of Washington, Seattle Seminaire, Institut de Mathématiques de Toulouse 5 March 2012 Seminaire, Toulouse Based on joint work
More informationShape constrained estimation: a brief introduction
Shape constrained estimation: a brief introduction Jon A. Wellner University of Washington, Seattle Cowles Foundation, Yale November 7, 2016 Econometrics lunch seminar, Yale Based on joint work with: Fadoua
More informationNonparametric estimation: s concave and log-concave densities: alternatives to maximum likelihood
Nonparametric estimation: s concave and log-concave densities: alternatives to maximum likelihood Jon A. Wellner University of Washington, Seattle Statistics Seminar, York October 15, 2015 Statistics Seminar,
More informationNonparametric estimation of. s concave and log-concave densities: alternatives to maximum likelihood
Nonparametric estimation of s concave and log-concave densities: alternatives to maximum likelihood Jon A. Wellner University of Washington, Seattle Cowles Foundation Seminar, Yale University November
More informationChernoff s distribution is log-concave. But why? (And why does it matter?)
Chernoff s distribution is log-concave But why? (And why does it matter?) Jon A. Wellner University of Washington, Seattle University of Michigan April 1, 2011 Woodroofe Seminar Based on joint work with:
More informationLog-concave distributions: definitions, properties, and consequences
Log-concave distributions: definitions, properties, and consequences Jon A. Wellner University of Washington, Seattle; visiting Heidelberg Seminaire, Institut de Mathématiques de Toulouse 28 February 202
More informationMaximum likelihood estimation of a log-concave density based on censored data
Maximum likelihood estimation of a log-concave density based on censored data Dominic Schuhmacher Institute of Mathematical Statistics and Actuarial Science University of Bern Joint work with Lutz Dümbgen
More informationNONPARAMETRIC ESTIMATION OF MULTIVARIATE CONVEX-TRANSFORMED DENSITIES. By Arseni Seregin, and Jon A. Wellner, University of Washington
Submitted to the Annals of Statistics arxiv: math.st/0911.4151v1 NONPARAMETRIC ESTIMATION OF MULTIVARIATE CONVEX-TRANSFORMED DENSITIES By Arseni Seregin, and Jon A. Wellner, University of Washington We
More informationNonparametric estimation under Shape Restrictions
Nonparametric estimation under Shape Restrictions Jon A. Wellner University of Washington, Seattle Statistical Seminar, Frejus, France August 30 - September 3, 2010 Outline: Five Lectures on Shape Restrictions
More informationA Law of the Iterated Logarithm. for Grenander s Estimator
A Law of the Iterated Logarithm for Grenander s Estimator Jon A. Wellner University of Washington, Seattle Probability Seminar October 24, 2016 Based on joint work with: Lutz Dümbgen and Malcolm Wolff
More informationRegularization prescriptions and convex duality: density estimation and Renyi entropies
Regularization prescriptions and convex duality: density estimation and Renyi entropies Ivan Mizera University of Alberta Department of Mathematical and Statistical Sciences Edmonton, Alberta, Canada Linz,
More informationOPTIMISATION CHALLENGES IN MODERN STATISTICS. Co-authors: Y. Chen, M. Cule, R. Gramacy, M. Yuan
OPTIMISATION CHALLENGES IN MODERN STATISTICS Co-authors: Y. Chen, M. Cule, R. Gramacy, M. Yuan How do optimisation problems arise in Statistics? Let X 1,...,X n be independent and identically distributed
More informationLog Concavity and Density Estimation
Log Concavity and Density Estimation Lutz Dümbgen, Kaspar Rufibach, André Hüsler (Universität Bern) Geurt Jongbloed (Amsterdam) Nicolai Bissantz (Göttingen) I. A Consulting Case II. III. IV. Log Concave
More informationRecent progress in log-concave density estimation
Submitted to Statistical Science Recent progress in log-concave density estimation Richard J. Samworth University of Cambridge arxiv:1709.03154v1 [stat.me] 10 Sep 2017 Abstract. In recent years, log-concave
More informationComputing Confidence Intervals for Log-Concave Densities. by Mahdis Azadbakhsh Supervisors: Hanna Jankowski Xin Gao York University
Computing Confidence Intervals for Log-Concave Densities by Mahdis Azadbakhsh Supervisors: Hanna Jankowski Xin Gao York University August 2012 Contents 1 Introduction 1 1.1 Introduction..............................
More informationarxiv: v3 [math.st] 4 Jun 2018
Inference for the mode of a log-concave density arxiv:1611.1348v3 [math.st] 4 Jun 218 Charles R. Doss and Jon A. Wellner School of Statistics University of Minnesota Minneapolis, MN 55455 e-mail: cdoss@stat.umn.edu
More informationMEDDE: MAXIMUM ENTROPY DEREGULARIZED DENSITY ESTIMATION
MEDDE: MAXIMUM ENTROPY DEREGULARIZED DENSITY ESTIMATION ROGER KOENKER Abstract. The trouble with likelihood is not the likelihood itself, it s the log. 1. Introduction Suppose that you would like to estimate
More informationBi-s -Concave Distributions
Bi-s -Concave Distributions Jon A. Wellner (Seattle) Non- and Semiparametric Statistics In Honor of Arnold Janssen, On the occasion of his 65th Birthday Non- and Semiparametric Statistics: In Honor of
More informationMaximum likelihood estimation of a multidimensional logconcave
Maximum likelihood estimation of a multidimensional logconcave density Madeleine Cule and Richard Samworth University of Cambridge, UK and Michael Stewart University of Sydney, Australia Summary. Let X
More informationAnnalee Gomm Math 714: Assignment #2
Annalee Gomm Math 714: Assignment #2 3.32. Verify that if A M, λ(a = 0, and B A, then B M and λ(b = 0. Suppose that A M with λ(a = 0, and let B be any subset of A. By the nonnegativity and monotonicity
More informationStochastic Comparisons of Weighted Sums of Arrangement Increasing Random Variables
Portland State University PDXScholar Mathematics and Statistics Faculty Publications and Presentations Fariborz Maseeh Department of Mathematics and Statistics 4-7-2015 Stochastic Comparisons of Weighted
More informationECE 4400:693 - Information Theory
ECE 4400:693 - Information Theory Dr. Nghi Tran Lecture 8: Differential Entropy Dr. Nghi Tran (ECE-University of Akron) ECE 4400:693 Lecture 1 / 43 Outline 1 Review: Entropy of discrete RVs 2 Differential
More informationStrong log-concavity is preserved by convolution
Strong log-concavity is preserved by convolution Jon A. Wellner Abstract. We review and formulate results concerning strong-log-concavity in both discrete and continuous settings. Although four different
More informationMATH MEASURE THEORY AND FOURIER ANALYSIS. Contents
MATH 3969 - MEASURE THEORY AND FOURIER ANALYSIS ANDREW TULLOCH Contents 1. Measure Theory 2 1.1. Properties of Measures 3 1.2. Constructing σ-algebras and measures 3 1.3. Properties of the Lebesgue measure
More informationSTATISTICAL INFERENCE UNDER ORDER RESTRICTIONS LIMIT THEORY FOR THE GRENANDER ESTIMATOR UNDER ALTERNATIVE HYPOTHESES
STATISTICAL INFERENCE UNDER ORDER RESTRICTIONS LIMIT THEORY FOR THE GRENANDER ESTIMATOR UNDER ALTERNATIVE HYPOTHESES By Jon A. Wellner University of Washington 1. Limit theory for the Grenander estimator.
More informationMaximum Likelihood Estimation under Shape Constraints
Maximum Likelihood Estimation under Shape Constraints Hanna K. Jankowski June 2 3, 29 Contents 1 Introduction 2 2 The MLE of a decreasing density 3 2.1 Finding the Estimator..............................
More informationTheorem 2.1 (Caratheodory). A (countably additive) probability measure on a field has an extension. n=1
Chapter 2 Probability measures 1. Existence Theorem 2.1 (Caratheodory). A (countably additive) probability measure on a field has an extension to the generated σ-field Proof of Theorem 2.1. Let F 0 be
More informationMaximum likelihood estimation of a multidimensional logconcave
Maximum likelihood estimation of a multidimensional logconcave density Madeleine Cule and Richard Samworth University of Cambridge, UK and Michael Stewart University of Sydney, Australia Summary. Let X
More informationHomework 11. Solutions
Homework 11. Solutions Problem 2.3.2. Let f n : R R be 1/n times the characteristic function of the interval (0, n). Show that f n 0 uniformly and f n µ L = 1. Why isn t it a counterexample to the Lebesgue
More informationwith Current Status Data
Estimation and Testing with Current Status Data Jon A. Wellner University of Washington Estimation and Testing p. 1/4 joint work with Moulinath Banerjee, University of Michigan Talk at Université Paul
More informationMath 5051 Measure Theory and Functional Analysis I Homework Assignment 3
Math 551 Measure Theory and Functional Analysis I Homework Assignment 3 Prof. Wickerhauser Due Monday, October 12th, 215 Please do Exercises 3*, 4, 5, 6, 8*, 11*, 17, 2, 21, 22, 27*. Exercises marked with
More informationSTAT 7032 Probability Spring Wlodek Bryc
STAT 7032 Probability Spring 2018 Wlodek Bryc Created: Friday, Jan 2, 2014 Revised for Spring 2018 Printed: January 9, 2018 File: Grad-Prob-2018.TEX Department of Mathematical Sciences, University of Cincinnati,
More informationJournal of Statistical Software
JSS Journal of Statistical Software March 2011, Volume 39, Issue 6. http://www.jstatsoft.org/ logcondens: Computations Related to Univariate Log-Concave Density Estimation Lutz Dümbgen University of Bern
More informationis a Borel subset of S Θ for each c R (Bertsekas and Shreve, 1978, Proposition 7.36) This always holds in practical applications.
Stat 811 Lecture Notes The Wald Consistency Theorem Charles J. Geyer April 9, 01 1 Analyticity Assumptions Let { f θ : θ Θ } be a family of subprobability densities 1 with respect to a measure µ on a measurable
More informationThe main results about probability measures are the following two facts:
Chapter 2 Probability measures The main results about probability measures are the following two facts: Theorem 2.1 (extension). If P is a (continuous) probability measure on a field F 0 then it has a
More informationX n D X lim n F n (x) = F (x) for all x C F. lim n F n(u) = F (u) for all u C F. (2)
14:17 11/16/2 TOPIC. Convergence in distribution and related notions. This section studies the notion of the so-called convergence in distribution of real random variables. This is the kind of convergence
More informationMaximum likelihood: counterexamples, examples, and open problems. Jon A. Wellner. University of Washington. Maximum likelihood: p.
Maximum likelihood: counterexamples, examples, and open problems Jon A. Wellner University of Washington Maximum likelihood: p. 1/7 Talk at University of Idaho, Department of Mathematics, September 15,
More information18.175: Lecture 3 Integration
18.175: Lecture 3 Scott Sheffield MIT Outline Outline Recall definitions Probability space is triple (Ω, F, P) where Ω is sample space, F is set of events (the σ-algebra) and P : F [0, 1] is the probability
More informationAsymptotics for posterior hazards
Asymptotics for posterior hazards Pierpaolo De Blasi University of Turin 10th August 2007, BNR Workshop, Isaac Newton Intitute, Cambridge, UK Joint work with Giovanni Peccati (Université Paris VI) and
More informationEXPOSITORY NOTES ON DISTRIBUTION THEORY, FALL 2018
EXPOSITORY NOTES ON DISTRIBUTION THEORY, FALL 2018 While these notes are under construction, I expect there will be many typos. The main reference for this is volume 1 of Hörmander, The analysis of liner
More informationAround the Brunn-Minkowski inequality
Around the Brunn-Minkowski inequality Andrea Colesanti Technische Universität Berlin - Institut für Mathematik January 28, 2015 Summary Summary The Brunn-Minkowski inequality Summary The Brunn-Minkowski
More informationMath 117: Infinite Sequences
Math 7: Infinite Sequences John Douglas Moore November, 008 The three main theorems in the theory of infinite sequences are the Monotone Convergence Theorem, the Cauchy Sequence Theorem and the Subsequence
More information3 Integration and Expectation
3 Integration and Expectation 3.1 Construction of the Lebesgue Integral Let (, F, µ) be a measure space (not necessarily a probability space). Our objective will be to define the Lebesgue integral R fdµ
More informationEcon 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines
Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines Maximilian Kasy Department of Economics, Harvard University 1 / 37 Agenda 6 equivalent representations of the
More informationSolutions Final Exam May. 14, 2014
Solutions Final Exam May. 14, 2014 1. Determine whether the following statements are true or false. Justify your answer (i.e., prove the claim, derive a contradiction or give a counter-example). (a) (10
More informationd(x n, x) d(x n, x nk ) + d(x nk, x) where we chose any fixed k > N
Problem 1. Let f : A R R have the property that for every x A, there exists ɛ > 0 such that f(t) > ɛ if t (x ɛ, x + ɛ) A. If the set A is compact, prove there exists c > 0 such that f(x) > c for all x
More informationMcGill University Math 354: Honors Analysis 3
Practice problems McGill University Math 354: Honors Analysis 3 not for credit Problem 1. Determine whether the family of F = {f n } functions f n (x) = x n is uniformly equicontinuous. 1st Solution: The
More informationTheory of Probability Fall 2008
MIT OpenCourseWare http://ocw.mit.edu 8.75 Theory of Probability Fall 008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. Section Prekopa-Leindler inequality,
More informationOn convex least squares estimation when the truth is linear
Electronic Journal of Statistics Vol. 1 16 171 9 ISSN: 1935-754 DOI: 1.114/15-EJS198 On convex least squares estimation when the truth is linear Yining Chen and Jon A. Wellner London School of Economics
More informationLectures 22-23: Conditional Expectations
Lectures 22-23: Conditional Expectations 1.) Definitions Let X be an integrable random variable defined on a probability space (Ω, F 0, P ) and let F be a sub-σ-algebra of F 0. Then the conditional expectation
More informationChapter 1. Optimality Conditions: Unconstrained Optimization. 1.1 Differentiable Problems
Chapter 1 Optimality Conditions: Unconstrained Optimization 1.1 Differentiable Problems Consider the problem of minimizing the function f : R n R where f is twice continuously differentiable on R n : P
More informationh(x) lim H(x) = lim Since h is nondecreasing then h(x) 0 for all x, and if h is discontinuous at a point x then H(x) > 0. Denote
Real Variables, Fall 4 Problem set 4 Solution suggestions Exercise. Let f be of bounded variation on [a, b]. Show that for each c (a, b), lim x c f(x) and lim x c f(x) exist. Prove that a monotone function
More informationStatic Problem Set 2 Solutions
Static Problem Set Solutions Jonathan Kreamer July, 0 Question (i) Let g, h be two concave functions. Is f = g + h a concave function? Prove it. Yes. Proof: Consider any two points x, x and α [0, ]. Let
More informationApproximation of Heavy-tailed distributions via infinite dimensional phase type distributions
1 / 36 Approximation of Heavy-tailed distributions via infinite dimensional phase type distributions Leonardo Rojas-Nandayapa The University of Queensland ANZAPW March, 2015. Barossa Valley, SA, Australia.
More informationLearning and Testing Structured Distributions
Learning and Testing Structured Distributions Ilias Diakonikolas University of Edinburgh Simons Institute March 2015 This Talk Algorithmic Framework for Distribution Estimation: Leads to fast & robust
More informationMath 328 Course Notes
Math 328 Course Notes Ian Robertson March 3, 2006 3 Properties of C[0, 1]: Sup-norm and Completeness In this chapter we are going to examine the vector space of all continuous functions defined on the
More informationStatistics 612: L p spaces, metrics on spaces of probabilites, and connections to estimation
Statistics 62: L p spaces, metrics on spaces of probabilites, and connections to estimation Moulinath Banerjee December 6, 2006 L p spaces and Hilbert spaces We first formally define L p spaces. Consider
More informationReview of measure theory
209: Honors nalysis in R n Review of measure theory 1 Outer measure, measure, measurable sets Definition 1 Let X be a set. nonempty family R of subsets of X is a ring if, B R B R and, B R B R hold. bove,
More informationLecture 3: Expected Value. These integrals are taken over all of Ω. If we wish to integrate over a measurable subset A Ω, we will write
Lecture 3: Expected Value 1.) Definitions. If X 0 is a random variable on (Ω, F, P), then we define its expected value to be EX = XdP. Notice that this quantity may be. For general X, we say that EX exists
More informationEstimation of a k monotone density, part 3: limiting Gaussian versions of the problem; invelopes and envelopes
Estimation of a monotone density, part 3: limiting Gaussian versions of the problem; invelopes and envelopes Fadoua Balabdaoui 1 and Jon A. Wellner University of Washington September, 004 Abstract Let
More informationAnalogy Principle. Asymptotic Theory Part II. James J. Heckman University of Chicago. Econ 312 This draft, April 5, 2006
Analogy Principle Asymptotic Theory Part II James J. Heckman University of Chicago Econ 312 This draft, April 5, 2006 Consider four methods: 1. Maximum Likelihood Estimation (MLE) 2. (Nonlinear) Least
More informationAnother Riesz Representation Theorem
Another Riesz Representation Theorem In these notes we prove (one version of) a theorem known as the Riesz Representation Theorem. Some people also call it the Riesz Markov Theorem. It expresses positive
More informationKernel Density Estimation
EECS 598: Statistical Learning Theory, Winter 2014 Topic 19 Kernel Density Estimation Lecturer: Clayton Scott Scribe: Yun Wei, Yanzhen Deng Disclaimer: These notes have not been subjected to the usual
More informationBayesian estimation of the discrepancy with misspecified parametric models
Bayesian estimation of the discrepancy with misspecified parametric models Pierpaolo De Blasi University of Torino & Collegio Carlo Alberto Bayesian Nonparametrics workshop ICERM, 17-21 September 2012
More informationEFFICIENT MULTIVARIATE ENTROPY ESTIMATION WITH
EFFICIENT MULTIVARIATE ENTROPY ESTIMATION WITH HINTS OF APPLICATIONS TO TESTING SHAPE CONSTRAINTS Richard Samworth, University of Cambridge Joint work with Thomas B. Berrett and Ming Yuan Collaborators
More informationEC 521 MATHEMATICAL METHODS FOR ECONOMICS. Lecture 1: Preliminaries
EC 521 MATHEMATICAL METHODS FOR ECONOMICS Lecture 1: Preliminaries Murat YILMAZ Boğaziçi University In this lecture we provide some basic facts from both Linear Algebra and Real Analysis, which are going
More informationSome stochastic inequalities for weighted sums
Some stochastic inequalities for weighted sums Yaming Yu Department of Statistics University of California Irvine, CA 92697, USA yamingy@uci.edu Abstract We compare weighted sums of i.i.d. positive random
More informationOn Strong Unimodality of Multivariate Discrete Distributions
R u t c o r Research R e p o r t On Strong Unimodality of Multivariate Discrete Distributions Ersoy Subasi a Mine Subasi b András Prékopa c RRR 47-2004, DECEMBER, 2004 RUTCOR Rutgers Center for Operations
More information(1) Consider the space S consisting of all continuous real-valued functions on the closed interval [0, 1]. For f, g S, define
Homework, Real Analysis I, Fall, 2010. (1) Consider the space S consisting of all continuous real-valued functions on the closed interval [0, 1]. For f, g S, define ρ(f, g) = 1 0 f(x) g(x) dx. Show that
More informationOrder Statistics and Distributions
Order Statistics and Distributions 1 Some Preliminary Comments and Ideas In this section we consider a random sample X 1, X 2,..., X n common continuous distribution function F and probability density
More informationMAXIMUM LIKELIHOOD ESTIMATION OF A UNIMODAL PROBABILITY MASS FUNCTION
Statistica Sinica 26 (2016), 1061-1086 doi:http://dx.doi.org/10.5705/ss.202014.0088 MAXIMUM LIKELIHOOD ESTIMATION OF A UNIMODAL PROBABILITY MASS FUNCTION Fadoua Balabdaoui and Hanna Jankowski Université
More informationA note on the convex infimum convolution inequality
A note on the convex infimum convolution inequality Naomi Feldheim, Arnaud Marsiglietti, Piotr Nayar, Jing Wang Abstract We characterize the symmetric measures which satisfy the one dimensional convex
More informationNotes, March 4, 2013, R. Dudley Maximum likelihood estimation: actual or supposed
18.466 Notes, March 4, 2013, R. Dudley Maximum likelihood estimation: actual or supposed 1. MLEs in exponential families Let f(x,θ) for x X and θ Θ be a likelihood function, that is, for present purposes,
More informationSteiner s formula and large deviations theory
Steiner s formula and large deviations theory Venkat Anantharam EECS Department University of California, Berkeley May 19, 2015 Simons Conference on Networks and Stochastic Geometry Blanton Museum of Art
More informationRANDOM WALKS WHOSE CONCAVE MAJORANTS OFTEN HAVE FEW FACES
RANDOM WALKS WHOSE CONCAVE MAJORANTS OFTEN HAVE FEW FACES ZHIHUA QIAO and J. MICHAEL STEELE Abstract. We construct a continuous distribution G such that the number of faces in the smallest concave majorant
More informationThe Caratheodory Construction of Measures
Chapter 5 The Caratheodory Construction of Measures Recall how our construction of Lebesgue measure in Chapter 2 proceeded from an initial notion of the size of a very restricted class of subsets of R,
More informationSTAT 512 sp 2018 Summary Sheet
STAT 5 sp 08 Summary Sheet Karl B. Gregory Spring 08. Transformations of a random variable Let X be a rv with support X and let g be a function mapping X to Y with inverse mapping g (A = {x X : g(x A}
More informationThe International Journal of Biostatistics
The International Journal of Biostatistics Volume 1, Issue 1 2005 Article 3 Score Statistics for Current Status Data: Comparisons with Likelihood Ratio and Wald Statistics Moulinath Banerjee Jon A. Wellner
More informationGeometry and Optimization of Relative Arbitrage
Geometry and Optimization of Relative Arbitrage Ting-Kam Leonard Wong joint work with Soumik Pal Department of Mathematics, University of Washington Financial/Actuarial Mathematic Seminar, University of
More informationEcon 508B: Lecture 5
Econ 508B: Lecture 5 Expectation, MGF and CGF Hongyi Liu Washington University in St. Louis July 31, 2017 Hongyi Liu (Washington University in St. Louis) Math Camp 2017 Stats July 31, 2017 1 / 23 Outline
More informationContinuity. Matt Rosenzweig
Continuity Matt Rosenzweig Contents 1 Continuity 1 1.1 Rudin Chapter 4 Exercises........................................ 1 1.1.1 Exercise 1............................................. 1 1.1.2 Exercise
More information3 Measurable Functions
3 Measurable Functions Notation A pair (X, F) where F is a σ-field of subsets of X is a measurable space. If µ is a measure on F then (X, F, µ) is a measure space. If µ(x) < then (X, F, µ) is a probability
More informationCHAPTER 6. Differentiation
CHPTER 6 Differentiation The generalization from elementary calculus of differentiation in measure theory is less obvious than that of integration, and the methods of treating it are somewhat involved.
More informationMath 5051 Measure Theory and Functional Analysis I Homework Assignment 2
Math 551 Measure Theory and Functional nalysis I Homework ssignment 2 Prof. Wickerhauser Due Friday, September 25th, 215 Please do Exercises 1, 4*, 7, 9*, 11, 12, 13, 16, 21*, 26, 28, 31, 32, 33, 36, 37.
More informationInformation Geometry
2015 Workshop on High-Dimensional Statistical Analysis Dec.11 (Friday) ~15 (Tuesday) Humanities and Social Sciences Center, Academia Sinica, Taiwan Information Geometry and Spontaneous Data Learning Shinto
More informationINDEPENDENT COMPONENT ANALYSIS VIA
INDEPENDENT COMPONENT ANALYSIS VIA NONPARAMETRIC MAXIMUM LIKELIHOOD ESTIMATION Truth Rotate S 2 2 1 0 1 2 3 4 X 2 2 1 0 1 2 3 4 4 2 0 2 4 6 4 2 0 2 4 6 S 1 X 1 Reconstructe S^ 2 2 1 0 1 2 3 4 Marginal
More informationMaximum likelihood: counterexamples, examples, and open problems
Maximum likelihood: counterexamples, examples, and open problems Jon A. Wellner University of Washington visiting Vrije Universiteit, Amsterdam Talk at BeNeLuxFra Mathematics Meeting 21 May, 2005 Email:
More informationQuick Tour of Basic Probability Theory and Linear Algebra
Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra CS224w: Social and Information Network Analysis Fall 2011 Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra Outline Definitions
More informationStatistica Sinica Preprint No: SS R2
Statistica Sinica Preprint No: SS-2014-0088R2 Title Maximum likelihood estimation of a unimodal probability mass function Manuscript ID SS-2014-0088R2 URL http://www.stat.sinica.edu.tw/statistica/ DOI
More informationSHAPE CONSTRAINED DENSITY ESTIMATION VIA PENALIZED RÉNYI DIVERGENCE
SHAPE CONSTRAINED DENSITY ESTIMATION VIA PENALIZED RÉNYI DIVERGENCE ROGER KOENKER AND IVAN MIZERA Abstract. Shape constraints play an increasingly prominent role in nonparametric function estimation. While
More informationLecture 1: Entropy, convexity, and matrix scaling CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: January 24, 2016
Lecture 1: Entropy, convexity, and matrix scaling CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: January 24, 2016 1 Entropy Since this course is about entropy maximization,
More informationNonparametric estimation under shape constraints, part 2
Nonparametric estimation under shape constraints, part 2 Piet Groeneboom, Delft University August 7, 2013 What to expect? Theory and open problems for interval censoring, case 2. Same for the bivariate
More informationLearning Multivariate Log-concave Distributions
Proceedings of Machine Learning Research vol 65:1 17, 2017 Learning Multivariate Log-concave Distributions Ilias Diakonikolas University of Southern California Daniel M. Kane University of California,
More informationIntegration on Measure Spaces
Chapter 3 Integration on Measure Spaces In this chapter we introduce the general notion of a measure on a space X, define the class of measurable functions, and define the integral, first on a class of
More informationProbability and Measure
Part II Year 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2018 84 Paper 4, Section II 26J Let (X, A) be a measurable space. Let T : X X be a measurable map, and µ a probability
More information1. Let A R be a nonempty set that is bounded from above, and let a be the least upper bound of A. Show that there exists a sequence {a n } n N
Applied Analysis prelim July 15, 216, with solutions Solve 4 of the problems 1-5 and 2 of the problems 6-8. We will only grade the first 4 problems attempted from1-5 and the first 2 attempted from problems
More informationarxiv: v2 [math.st] 21 Jan 2018
Maximum a Posteriori Estimators as a Limit of Bayes Estimators arxiv:1611.05917v2 [math.st] 21 Jan 2018 Robert Bassett Mathematics Univ. California, Davis rbassett@math.ucdavis.edu Julio Deride Mathematics
More informationAsymptotics for posterior hazards
Asymptotics for posterior hazards Igor Prünster University of Turin, Collegio Carlo Alberto and ICER Joint work with P. Di Biasi and G. Peccati Workshop on Limit Theorems and Applications Paris, 16th January
More informationSolutions to Tutorial 11 (Week 12)
THE UIVERSITY OF SYDEY SCHOOL OF MATHEMATICS AD STATISTICS Solutions to Tutorial 11 (Week 12) MATH3969: Measure Theory and Fourier Analysis (Advanced) Semester 2, 2017 Web Page: http://sydney.edu.au/science/maths/u/ug/sm/math3969/
More information