Nonparametric Estimation of Luminosity Functions
|
|
- Ralf Snow
- 6 years ago
- Views:
Transcription
1 x x Nonparametric Estimation of Luminosity Functions Chad Schafer Department of Statistics, Carnegie Mellon University cschafer@stat.cmu.edu 1
2 Luminosity Functions The luminosity function gives the number of (quasars, galaxies, galaxy clusters,...) as a function of luminosity (L) or absolute magnitude (M) The 2-d luminosity function allows for dependence on redshift (z) Schechter Form (Schechter, 1976): ( ) ( L φ S (L) dl = n exp L ) α ( L d L L or φ S (M) dm n (α+1)(M M) exp ( 10 ) 0.4(M M) dm L ) 2
3 Schechter (1976) 3
4 From Binney and Merrifield (1998), pages : This formula was initially motivated by a simple model of galaxy formation (Press and Schecter, 1974), but it has proved to have a wider range of application than originally envisaged... With larger, deeper surveys, the limitations of the simple Schechter function start to become apparent. Particular problems on faint end, and with luminosity function evolution with redshift. Blanton, et al. (2003): Model 2-d galaxy luminosity function as Φ(M, z) = n10 0.4(z z 0)P K k=1 Φ k 2πσ 2 M exp [ 1 2 ] (M M k + (z z 0 ) Q) 2 σ 2 M 4
5 x Quasars Among the furthest observable objects Quasar PKS 2349, Observed by Hubble Telescope in 1996 Image courtesy of NASA 5
6 Absolute Magnitude Redshift Sample of 15,057 quasars from Sloan Digital Sky Survey. (Richards et al. 2006) 6
7 Luminosity Functions as Probability Density Functions Integrating a luminosity function gives the count in that bin count with 22 M 21 = φ(m) dm Proportional to the probability a randomly chosen such object is such that 22 M 21. Hence, estimating φ(m) analogous to density estimation 7
8 Density Estimation Basic Problem: Estimate f, where assume that X 1,X 2,...,X n f, i.e., for a b, P(a X i b) = b a f(x) dx Parametric Approach: Assume f belongs to some family of densities (e.g., Gaussian), and estimate the unknown parameters via maximum likelihood Nonparametric Approach: Estimate built on smoothing available data, slower rate of convergence, but less bias The histogram is a simple example of a nonparametric density estimator. 8
9 xxx Density Estimation Sample of size 303: Note that x = 20.18, s =
10 xxx Parametric Estimator density Latitude of Initial Point Assuming Gaussian, estimate µ = and σ =
11 Parametric Estimator Density Latitude of Initial Point Compared with the histogram estimate of the density. 11
12 Advantages of Parametric Form Relatively simple to fit, via maximum likelihood Functional form Easy calculation of probabilities Smaller errors, on average, provided assumption regarding density is correct 12
13 xxx Errors in Density Estimators Error at one value x: (f(x) f(x)) 2 Error accumulated over all x: ISE = (f(x) f(x)) 2 dx Mean Integrated Squared Error (MISE): ( ) MISE = E (f(x) f(x)) 2 dx 13
14 The Bias Variance Tradeoff Can write MISE = b 2 (x)dx + v(x)dx where the bias at x is and the variance at x is b(x) = E( f(x)) f(x) v(x) = V( f(x)) 14
15 The Bias Variance Tradeoff MSE Bias squared Variance Less Optimal smoothing More 15
16 Histograms Create bins B 1,B 2,...,B m of width h. Define f n (x) = m j=1 p j h I(x B j). where p j is proportion of observations in B j. Note that f n (x)dx = 1. A histogram is an example of a nonparametric density estimator. Note that it is controlled by the tuning parameter h, the bin width. 16
17 Histograms Density Latitude of Initial Point Seems like too many bins, i.e. h is too small. 17
18 Histograms Density Latitude of Initial Point Seems like too few bins, i.e. h is too large. 18
19 xxx Histograms Latitude of Initial Point Density Seems better. 19
20 Histograms Density Latitude of Initial Point Clear that normal curve is not a good fit. 20
21 Histograms For histograms, ( MISE = E ) (f(x) f(x)) 2 dx h2 12 (f (u)) 2 du + 1 nh The value h that minimizes this is h = 1 ( 1/3 6 n 1/3 (f. (u)) du) 2 and then MISE C 2 n 2/3 21
22 Kernel Density Estimation We can improve histograms by introducing smoothing. f(x) = 1 n ( ) x Xi K. nh h Here, K( ) is the kernel function, itself a smooth density. i=1 22
23 Kernels A kernel is any smooth function K such that K(x) 0 and K(x)dx = 1, xk(x)dx = 0 and σk 2 x 2 K(x)dx > 0. Some commonly used kernels are the following: the boxcar kernel : K(x) = 1 I( x < 1), 2 the Gaussian kernel : K(x) = 1 e x2 /2, 2π the Epanechnikov kernel : K(x) = 3 4 (1 x2 )I( x < 1) the tricube kernel : K(x) = (1 x 3 ) 3 I( x < 1). 23
24 xxx Kernels
25 xxx Kernel Density Estimation Places a smoothed-out lump of mass of size 1/n over each data point The shape of the lumps is controlled by K( ); their width is controlled by h. 25
26 Kernel Density Estimation Density Latitude of Initial Point h chosen too large. 26
27 xxx Kernel Density Estimation Density Latitude of Initial Point h chosen too small. 27
28 Kernel Density Estimation Density Latitude of Initial Point Just right. 28
29 xxx Kernel Density Estimation MSE 1 4 c2 1h 4 (f (x)) 2 dx + K 2 (x)dx nh Optimal bandwidth is h = ( c2 c 2 1A(f)n ) 1/5 where c 1 = x 2 K(x)dx, c 2 = K(x) 2 dx and A(f) = (f (x)) 2 dx. Then, MSE C 3 n 4/5. 29
30 The Bias Variance Tradeoff For many smoothers: which is minimized at MISE c 1 h 4 + c 2 nh h = O ( 1 n 1/5 ) Hence, MISE = O whereas, for parametric problems MISE = O ( 1 n 4/5 ( ) 1 n ) 30
31 xxx Comparisons So, for parametric form, if it s correct, MISE C 1 n For histograms, MISE C 2 n 2/3 For estimators based on smoothing, MISE C 3 n 4/5 31
32 xxx Comparisons f(x) truth closest normal x Truth is standard normal. 32
33 Comparisons MSE 0e+00 1e 05 2e 05 3e 05 4e sample size (n) normal kernel Assuming normality has its advantages. 33
34 Comparisons truth closest normal f(x) Truth is t-distribution with 5 degrees of freedom. x 34
35 Comparisons normal kernel MSE 5.0e e e sample size (n) Normal assumption costs, even for moderate sample size. 35
36 Cross-Validation Objective, data-driven choice of h is possible: ( ) MISE = E (f(x) f(x)) 2 dx ( = E = J(h) + ) f(x) 2 dx f(x) 2 dx ( 2E ) f(x)f(x)dx + f(x) 2 dx Unbiased Estimator of J(h): Ĵ(h) = ( ) 2 2 fn (x) dx n n i=1 f ( i) (X i ) where f ( i) is the density estimator obtained after removing the i th observation. 36
37 x Local Density Estimation The idea of local modelling (recal local polynomial regression) extends to density estimation. The local likelihood density estimator developed independently by Loader (1996) and Hjort and Jones (1996) allows for fitting of nonparametric models of this type. 37
38 Local Density Estimation Model assumes that for u near x, log(f(x)) a 0 + a 1 (u x) + + a p (u x) p Thus, ( ) log f(x) = â 0 + â 1 (u x) + + â p (u x) p is the local (near x) estimate of log( f( )) Estimates obtained by maximizing kernel-weigthed likelihood Repeated for a grid of values x x x x 38
39 Log Density A few local linear estimates of log( f( )) 39
40 Log Density Smoothed toghether to get log( f( )) 40
41 Back to Luminosity Functions... Simple parametric forms not realistic for large data sets Could build more complex parametric forms, e.g., Blanton, et al. (2003) Φ(M, z) = n10 0.4(z z 0)P K k=1 Φ k 2πσ 2 M exp [ 1 2 ] (M M k + (z z 0 ) Q) 2 σ 2 M Nonparametric approaches need to account for observational limitations, distinguish physical variations from observational variations 41
42 Truncation Suppose that X 1,X 2,...,X m f, but data are truncated: only get to see those X i such that X i k. Formally, we get to observe X 1,X 2,...,X n f, where f(x) f R k (x) = f(x)dx, if x k 0, if x > k Clearly, must take truncation into account when using the observed data to estimate f. 42
43 Absolute Magnitude Redshift Sample of 15,057 quasars from Sloan Digital Sky Survey. (Richards et al. 2006) 43
44 1/V max Estimator (Schmidt, 1968; Eales, 1993) Observe objects of absolute magnitudes M 1,M 2,...,M n Divide absolute magnitude range into bins. Estimate φ over j th bin as bin j φ(m) = i bin j 1 V max (i) where V max (i) is the volume over which an object of magnitude M i can be detected by the survey This is a weighted histogram 44
45 Nonparametric Maximum Likelihood (NPMLE) Lynden-Bell (1971): Consider the case of one-sided truncation Efron and Petrosian (1999): Two-sided (double) truncation Know that NPMLE will assign all probability to oberved values, but they will not be equal probabilities NPMLE estimators for this case are implemented in R package DTDA. 45
46 NPMLE Examples Data Value Observation Number Vertical lines are truncation ranges, dots are data values. 46
47 NPMLE Examples Data Value Observation Number The probability assigned to each by the NPMLE. 47
48 NPMLE Examples Data Value Observation Number Differential truncation. 48
49 NPMLE Examples Data Value Observation Number Both of the intervals (, 2) and (2, ) has probability
50 NPMLE Examples Data Value Observation Number An example where it performs poorly. 50
51 NPMLE Examples Data Value Observation Number xxx 51
52 NPMLE Examples Data Value Observation Number Example from Efron and Petrosian (1999) 52
53 NPMLE Examples Data Value Observation Number 53
54 Efron and Petrosian (1999) Quasar Example 54
55 Efron and Petrosian (1999) Quasar Example 55
56 Efron and Petrosian (1999) Quasar Example The SEF fit is a special exponential family model that assumes that log f(x) = β 0 + β 1 x + β 2 x 2 + β 3 x 3 This model can be fit via maximum likelihood. BUT: Each of these fits assumes independence between redshift and luminosity, i.e., no evolution of the luminosity function with redshift. 56
57 Absolute Magnitude Redshift Sample of 15,057 quasars from Sloan Digital Sky Survey. (Richards et al. 2006) 57
58 The Challenge: Model selection Goal: Given the truncated data shown in the scatter plot, estimate the bivariate density (φ(z,m)). Too Strong: Assume independence: φ(z, M) = f(z)g(m), or assume parametric form: φ(z,m,θ) Too Weak: Use fully nonparametric approach 58
59 xxx A Semiparametric Model Schafer (2007) We utilize a semiparametric model which allows one to place minimal assumptions on the form of the bivariate density and greatly diminish bias due to truncation and boundary effects. Decompose the bivariate density φ(z,m) into log(φ(z,m)) = f(z) + g(m) + h(z,m,θ) where f(z) and g(m) are estimated using nonparametric methods and h(z,m,θ) will take an assumed parametric form; it is intended to model the dependence between the two random variables. We use h(z,m,θ) = θzm, but another choice could be made, e.g. motivated by a physical theory. 59
60 Fitting the model: The motivating (naive) idea Assume independence and write φ(z, M) = f(z)g(m) where f( ) is the density for redshift and g( ) is the density for absolute magnitude. x Absolute Magnitude A A(4.0,.) x Let A denote the region outside of which of the data are truncated and let A(z, ) denote the cross-section of A at z. Assume, for the moment, that g( ) is known Redshift 60
61 x The available data allow for estimation of f (z) φ(z,m)dm = f(z) A(z, ) A(z, ) g(m)dm for all z, since it is the density of the observable redshift. Assuming for the moment that g( ) were known, it is possible to estimate f( ) using f(z) = f / ( ) (z) g(m)dm. A(z, ) Starting with an initial guess at g( ), we could iterate between assuming g( ) is known, and estimating f( ), and vice versa. But, the variance of this estimator could be huge: Consider the behavior of f(z) for z where A(z, ) g(m)dm is small. 61
62 Absolute Magnitude Redshift Depicting A(1.5, ) 62
63 Density % Unobserved 24% Observed Absolute Magnitude 24% of area under assumed g( ) falls in A(1.5, ) 63
64 Percent Observed Redshift (z) Percent of area under g( ) in A(z, ), as a function of z. 64
65 Estimated Redshift Density Redshift Smooth f ( ) does not lead to smooth f( ) 65
66 x Local density estimation Instead, we write ( log(f (z)) = log(f(z)) + log A(z, ) ) g(m)dm and treat ( log A(z, ) ) g(m)dm as an offset term in a model for log(f (z)) and consider only forms for f( ) which are controlled by smoothing parameters chosen by the user. The local likelihood density estimator developed independently by Loader (1996) and Hjort and Jones (1996) allows for fitting of nonparametric models of this type; the offset is easily incorporated. 66
67 x M-estimators This formulation allows the estimation problem to be couched as an M-estimator, i.e. choose parameters of the local models to maximize criterion n ψ(z i,m i ) i=1 Leads to estimates of standard errors, etc. The parametric component h(z,m,θ) is also easily included in the offset. Can use weights allows for varying selection function: n w i ψ(z i,m i ) i=1 67
68 Bandwidth Selection via Cross-Validation One ideal choice of the smoothing parameters would minimize the integrated mean squared error: [ ) ] 2 E ( φ(z,m) φ(z,m) dz dm. A An unbiased estimator of the integrated mean squared error is φ 2 (z,m)dz dm 2 n φ ( j) (z j,m j ) n A where φ ( j) (z j,m j ) denotes the estimate found when omitting the j th data pair from the analysis efficiently estimated here. j=1 68
69 Absolute Magnitude Redshift Sample of 15,057 quasars from the Sloan Digital Sky Survey. 69
70 Absolute Magnitude Redshift Same plot, with dots scaled by value of selection function for that observation. 70
71 Key aspects of the approach Allows for the removal of the independence assumption, without assuming a parametric form Iterative algorithm not computationally burdensome; must converge to a unique estimate Statistical theory leads to good approximations to standard errors for all estimated quantities Cross-validation estimator of integrated mean squared error is easily calculated, leading to good bandwidth selection Natural incorporation of the selection function 71
72 LSCV λ z λ M = 0.15 λ M = 0.16 λ M = 0.17 λ M = 0.18 λ M = 0.19 Finding the optimal bandwidths. 72
73 Absolute Magnitude Redshift Estimate of bivariate density, i.e. 2-D luminosity function. 73
74 log(ρ(z))(m < )(Mpc 3 ) Redshift Marginal density for redshift. 74
75 Φ(M i ) (Mpc 3 mag 1 ) z = 0.49 z = 1.25 z = 2.01 Φ(M i ) (Mpc 3 mag 1 ) z = 2.8 z = 3.75 z = M i M i M i Slices of the bivariate estimate, compared with Richards, et al. (2006). 75
76 z = 1 z = Φ(M) (Mpc 3 mag 1 ) Φ(M) (Mpc 3 mag 1 ) z = 3 z = M M Estimates based on simulated data. 76
77 Nonparametric Bayesian Approach Kelly et al. (2008): Model 2-d luminosity function as mixture of Gaussians. Parameterized form allows for Bayesian inference Natural uncertainty measures for any functions of the luminosity function 77
78 Figure 3(a) from Kelly, et al. (2008) 78
79 Conclusion Luminosity functions as probability densities The potential of nonparametric density estimation Moving beyond the independence assumption and binning 79
80 References Binney and Merrifield (1998). Galactic Astronomy. Princeton University Press. Blanton (2003). ApJ. 592, Eales (1993). ApJ Efron and Petrosian (1999). JASA Kelly, et al. (2008). ApJ Lynden-Bell (1971). MNRAS Press and Schechter (1974). ApJ 187, Richards et al. (2006). AJ. 131, Schafer (2007). ApJ 661, Schechter (1976). ApJ (203), Schmidt (1968). ApJ (151), Takeuchi, et al. (2000). ApJS (129),
81 Acknowledgements Peter Freeman, Chris Genovese, and Larry Wasserman, Department of Statistics, CMU. Sloan Digital Sky Survey Research supported by NSF grants # and # For more information... See cschafer/bivtrunc Contact Chad at 81
Nonparametric Statistics
Nonparametric Statistics Jessi Cisewski Yale University Astrostatistics Summer School - XI Wednesday, June 3, 2015 1 Overview Many of the standard statistical inference procedures are based on assumptions
More informationarxiv:astro-ph/ v1 15 Feb 2007
A Statistical Method for Estimating Luminosity Functions using Truncated Data Chad M. Schafer cschafer@stat.cmu.edu arxiv:astro-ph/0702401v1 15 Feb 2007 Department of Statistics, Carnegie Mellon University
More informationImproved Astronomical Inferences via Nonparametric Density Estimation
Improved Astronomical Inferences via Nonparametric Density Estimation Chad M. Schafer, InCA Group www.incagroup.org Department of Statistics Carnegie Mellon University Work Supported by NSF, NASA-AISR
More informationIntroduction to Regression
Introduction to Regression p. 1/97 Introduction to Regression Chad Schafer cschafer@stat.cmu.edu Carnegie Mellon University Introduction to Regression p. 1/97 Acknowledgement Larry Wasserman, All of Nonparametric
More informationIntroduction to Regression
Introduction to Regression Chad M. Schafer cschafer@stat.cmu.edu Carnegie Mellon University Introduction to Regression p. 1/100 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression
More informationIntroduction to Regression
Introduction to Regression David E Jones (slides mostly by Chad M Schafer) June 1, 2016 1 / 102 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression Nonparametric Procedures
More informationIntroduction to Regression
Introduction to Regression Chad M. Schafer May 20, 2015 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression Nonparametric Procedures Cross Validation Local Polynomial Regression
More informationAddressing the Challenges of Luminosity Function Estimation via Likelihood-Free Inference
Addressing the Challenges of Luminosity Function Estimation via Likelihood-Free Inference Chad M. Schafer www.stat.cmu.edu/ cschafer Department of Statistics Carnegie Mellon University June 2011 1 Acknowledgements
More informationStatistics: Learning models from data
DS-GA 1002 Lecture notes 5 October 19, 2015 Statistics: Learning models from data Learning models from data that are assumed to be generated probabilistically from a certain unknown distribution is a crucial
More informationGalaxy Morphology Refresher. Driver et al. 2006
Galaxy Morphology Refresher Driver et al. 2006 http://www.galaxyzoo.org Galaxy Luminosity Functions Luminosity Function is the number density (# per volume) of some population of objects (e.g., galaxies)
More informationgroup. redshifts. However, the sampling is fairly complete out to about z = 0.2.
Non parametric Inference in Astrophysics Larry Wasserman, Chris Miller, Bob Nichol, Chris Genovese, Woncheol Jang, Andy Connolly, Andrew Moore, Jeff Schneider and the PICA group. 1 We discuss non parametric
More informationQuantitative Economics for the Evaluation of the European Policy. Dipartimento di Economia e Management
Quantitative Economics for the Evaluation of the European Policy Dipartimento di Economia e Management Irene Brunetti 1 Davide Fiaschi 2 Angela Parenti 3 9 ottobre 2015 1 ireneb@ec.unipi.it. 2 davide.fiaschi@unipi.it.
More informationSpace Telescope Science Institute statistics mini-course. October Inference I: Estimation, Confidence Intervals, and Tests of Hypotheses
Space Telescope Science Institute statistics mini-course October 2011 Inference I: Estimation, Confidence Intervals, and Tests of Hypotheses James L Rosenberger Acknowledgements: Donald Richards, William
More information12 - Nonparametric Density Estimation
ST 697 Fall 2017 1/49 12 - Nonparametric Density Estimation ST 697 Fall 2017 University of Alabama Density Review ST 697 Fall 2017 2/49 Continuous Random Variables ST 697 Fall 2017 3/49 1.0 0.8 F(x) 0.6
More informationChapter 9. Non-Parametric Density Function Estimation
9-1 Density Estimation Version 1.1 Chapter 9 Non-Parametric Density Function Estimation 9.1. Introduction We have discussed several estimation techniques: method of moments, maximum likelihood, and least
More informationNonparametric Methods
Nonparametric Methods Michael R. Roberts Department of Finance The Wharton School University of Pennsylvania July 28, 2009 Michael R. Roberts Nonparametric Methods 1/42 Overview Great for data analysis
More informationO Combining cross-validation and plug-in methods - for kernel density bandwidth selection O
O Combining cross-validation and plug-in methods - for kernel density selection O Carlos Tenreiro CMUC and DMUC, University of Coimbra PhD Program UC UP February 18, 2011 1 Overview The nonparametric problem
More informationEcon 582 Nonparametric Regression
Econ 582 Nonparametric Regression Eric Zivot May 28, 2013 Nonparametric Regression Sofarwehaveonlyconsideredlinearregressionmodels = x 0 β + [ x ]=0 [ x = x] =x 0 β = [ x = x] [ x = x] x = β The assume
More informationCensoring and truncation in astronomical surveys. Eric Feigelson (Astronomy & Astrophysics, Penn State)
Censoring and truncation in astronomical surveys Eric Feigelson (Astronomy & Astrophysics, Penn State) Outline 1. Examples of astronomical surveys 2. One survey: Truncation and spatial/luminosity distributions
More informationMeasurement Error and Linear Regression of Astronomical Data. Brandon Kelly Penn State Summer School in Astrostatistics, June 2007
Measurement Error and Linear Regression of Astronomical Data Brandon Kelly Penn State Summer School in Astrostatistics, June 2007 Classical Regression Model Collect n data points, denote i th pair as (η
More informationChapter 9. Non-Parametric Density Function Estimation
9-1 Density Estimation Version 1.2 Chapter 9 Non-Parametric Density Function Estimation 9.1. Introduction We have discussed several estimation techniques: method of moments, maximum likelihood, and least
More informationHistogram Härdle, Müller, Sperlich, Werwatz, 1995, Nonparametric and Semiparametric Models, An Introduction
Härdle, Müller, Sperlich, Werwatz, 1995, Nonparametric and Semiparametric Models, An Introduction Tine Buch-Kromann Construction X 1,..., X n iid r.v. with (unknown) density, f. Aim: Estimate the density
More informationKernel density estimation with doubly truncated data
Kernel density estimation with doubly truncated data Jacobo de Uña-Álvarez jacobo@uvigo.es http://webs.uvigo.es/jacobo (joint with Carla Moreira) Department of Statistics and OR University of Vigo Spain
More informationDensity estimation Nonparametric conditional mean estimation Semiparametric conditional mean estimation. Nonparametrics. Gabriel Montes-Rojas
0 0 5 Motivation: Regression discontinuity (Angrist&Pischke) Outcome.5 1 1.5 A. Linear E[Y 0i X i] 0.2.4.6.8 1 X Outcome.5 1 1.5 B. Nonlinear E[Y 0i X i] i 0.2.4.6.8 1 X utcome.5 1 1.5 C. Nonlinearity
More informationPATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS Parametric Distributions Basic building blocks: Need to determine given Representation: or? Recall Curve Fitting Binary Variables
More informationNonparametric Inference in Cosmology and Astrophysics: Biases and Variants
Nonparametric Inference in Cosmology and Astrophysics: Biases and Variants Christopher R. Genovese Department of Statistics Carnegie Mellon University http://www.stat.cmu.edu/ ~ genovese/ Collaborators:
More informationSparse Nonparametric Density Estimation in High Dimensions Using the Rodeo
Sparse Nonparametric Density Estimation in High Dimensions Using the Rodeo Han Liu John Lafferty Larry Wasserman Statistics Department Computer Science Department Machine Learning Department Carnegie Mellon
More informationMinimum Hellinger Distance Estimation in a. Semiparametric Mixture Model
Minimum Hellinger Distance Estimation in a Semiparametric Mixture Model Sijia Xiang 1, Weixin Yao 1, and Jingjing Wu 2 1 Department of Statistics, Kansas State University, Manhattan, Kansas, USA 66506-0802.
More informationIntensity Analysis of Spatial Point Patterns Geog 210C Introduction to Spatial Data Analysis
Intensity Analysis of Spatial Point Patterns Geog 210C Introduction to Spatial Data Analysis Chris Funk Lecture 4 Spatial Point Patterns Definition Set of point locations with recorded events" within study
More informationConfidence intervals for kernel density estimation
Stata User Group - 9th UK meeting - 19/20 May 2003 Confidence intervals for kernel density estimation Carlo Fiorio c.fiorio@lse.ac.uk London School of Economics and STICERD Stata User Group - 9th UK meeting
More informationLog-Density Estimation with Application to Approximate Likelihood Inference
Log-Density Estimation with Application to Approximate Likelihood Inference Martin Hazelton 1 Institute of Fundamental Sciences Massey University 19 November 2015 1 Email: m.hazelton@massey.ac.nz WWPMS,
More informationDiscrete Mathematics and Probability Theory Fall 2015 Lecture 21
CS 70 Discrete Mathematics and Probability Theory Fall 205 Lecture 2 Inference In this note we revisit the problem of inference: Given some data or observations from the world, what can we infer about
More informationBayesian estimation of the discrepancy with misspecified parametric models
Bayesian estimation of the discrepancy with misspecified parametric models Pierpaolo De Blasi University of Torino & Collegio Carlo Alberto Bayesian Nonparametrics workshop ICERM, 17-21 September 2012
More informationDr Martin Hendry Dept of Physics and Astronomy University of Glasgow, UK
Dr Martin Hendry Dept of Physics and Astronomy University of Glasgow, UK martin@astroglaacuk Dr Martin Hendry, Russell Johnston Dept of Physics and Astronomy University of Glasgow, UK Who am I? William
More informationMotivational Example
Motivational Example Data: Observational longitudinal study of obesity from birth to adulthood. Overall Goal: Build age-, gender-, height-specific growth charts (under 3 year) to diagnose growth abnomalities.
More informationPattern Recognition and Machine Learning. Bishop Chapter 2: Probability Distributions
Pattern Recognition and Machine Learning Chapter 2: Probability Distributions Cécile Amblard Alex Kläser Jakob Verbeek October 11, 27 Probability Distributions: General Density Estimation: given a finite
More informationGaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012
Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature
More informationStatistics notes. A clear statistical framework formulates the logic of what we are doing and why. It allows us to make precise statements.
Statistics notes Introductory comments These notes provide a summary or cheat sheet covering some basic statistical recipes and methods. These will be discussed in more detail in the lectures! What is
More informationExploiting Sparse Non-Linear Structure in Astronomical Data
Exploiting Sparse Non-Linear Structure in Astronomical Data Ann B. Lee Department of Statistics and Department of Machine Learning, Carnegie Mellon University Joint work with P. Freeman, C. Schafer, and
More informationBoundary Correction Methods in Kernel Density Estimation Tom Alberts C o u(r)a n (t) Institute joint work with R.J. Karunamuni University of Alberta
Boundary Correction Methods in Kernel Density Estimation Tom Alberts C o u(r)a n (t) Institute joint work with R.J. Karunamuni University of Alberta November 29, 2007 Outline Overview of Kernel Density
More informationNonparametric Inference and the Dark Energy Equation of State
Nonparametric Inference and the Dark Energy Equation of State Christopher R. Genovese Peter E. Freeman Larry Wasserman Department of Statistics Carnegie Mellon University http://www.stat.cmu.edu/ ~ genovese/
More informationGalaxy number counts in a presence of the graviton background
arxiv:astro-ph/0606223v3 15 Jun 2006 Galaxy number counts in a presence of the graviton background Michael A. Ivanov Physics Dept., Belarus State University of Informatics and Radioelectronics, 6 P. Brovka
More informationNonparametric Density Estimation
Nonparametric Density Estimation Econ 690 Purdue University Justin L. Tobias (Purdue) Nonparametric Density Estimation 1 / 29 Density Estimation Suppose that you had some data, say on wages, and you wanted
More informationAkaike Information Criterion to Select the Parametric Detection Function for Kernel Estimator Using Line Transect Data
Journal of Modern Applied Statistical Methods Volume 12 Issue 2 Article 21 11-1-2013 Akaike Information Criterion to Select the Parametric Detection Function for Kernel Estimator Using Line Transect Data
More informationBayes via forward simulation. Approximate Bayesian Computation
: Approximate Bayesian Computation Jessi Cisewski Carnegie Mellon University June 2014 What is Approximate Bayesian Computing? Likelihood-free approach Works by simulating from the forward process What
More informationTime Series and Forecasting Lecture 4 NonLinear Time Series
Time Series and Forecasting Lecture 4 NonLinear Time Series Bruce E. Hansen Summer School in Economics and Econometrics University of Crete July 23-27, 2012 Bruce Hansen (University of Wisconsin) Foundations
More informationAdaptive Nonparametric Density Estimators
Adaptive Nonparametric Density Estimators by Alan J. Izenman Introduction Theoretical results and practical application of histograms as density estimators usually assume a fixed-partition approach, where
More informationSparse Nonparametric Density Estimation in High Dimensions Using the Rodeo
Outline in High Dimensions Using the Rodeo Han Liu 1,2 John Lafferty 2,3 Larry Wasserman 1,2 1 Statistics Department, 2 Machine Learning Department, 3 Computer Science Department, Carnegie Mellon University
More informationThe flickering luminosity method
The flickering luminosity method Martin Feix in collaboration with Adi Nusser (Technion) and Enzo Branchini (Roma Tre) Institut d Astrophysique de Paris SSG16 Workshop, February 2nd 2016 Outline 1 Motivation
More informationAre Bayesian Inferences Weak for Wasserman s Example?
Are Bayesian Inferences Weak for Wasserman s Example? Longhai Li longhai@math.usask.ca Department of Mathematics and Statistics University of Saskatchewan Saskatoon, Saskatchewan, S7N 5E6 Canada Quebec
More informationIntroduction to machine learning and pattern recognition Lecture 2 Coryn Bailer-Jones
Introduction to machine learning and pattern recognition Lecture 2 Coryn Bailer-Jones http://www.mpia.de/homes/calj/mlpr_mpia2008.html 1 1 Last week... supervised and unsupervised methods need adaptive
More informationGalaxy Luminosity Function
Galaxy Luminosity Function Methods: Vmax Max Likelihood! Observations of LF: Shape of LF Field LF LF in Groups and Clusters Luminosity Function Φ(M)dM is the number of galaxies per unit volume with absolute
More informationEstimation of a Two-component Mixture Model
Estimation of a Two-component Mixture Model Bodhisattva Sen 1,2 University of Cambridge, Cambridge, UK Columbia University, New York, USA Indian Statistical Institute, Kolkata, India 6 August, 2012 1 Joint
More informationCurve Fitting Re-visited, Bishop1.2.5
Curve Fitting Re-visited, Bishop1.2.5 Maximum Likelihood Bishop 1.2.5 Model Likelihood differentiation p(t x, w, β) = Maximum Likelihood N N ( t n y(x n, w), β 1). (1.61) n=1 As we did in the case of the
More informationBasic Sampling Methods
Basic Sampling Methods Sargur Srihari srihari@cedar.buffalo.edu 1 1. Motivation Topics Intractability in ML How sampling can help 2. Ancestral Sampling Using BNs 3. Transforming a Uniform Distribution
More informationKernel density estimation
Kernel density estimation Patrick Breheny October 18 Patrick Breheny STA 621: Nonparametric Statistics 1/34 Introduction Kernel Density Estimation We ve looked at one method for estimating density: histograms
More informationCheng Soon Ong & Christian Walder. Canberra February June 2018
Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 2018 Outlines Overview Introduction Linear Algebra Probability Linear Regression
More informationLocal Polynomial Modelling and Its Applications
Local Polynomial Modelling and Its Applications J. Fan Department of Statistics University of North Carolina Chapel Hill, USA and I. Gijbels Institute of Statistics Catholic University oflouvain Louvain-la-Neuve,
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationA Process over all Stationary Covariance Kernels
A Process over all Stationary Covariance Kernels Andrew Gordon Wilson June 9, 0 Abstract I define a process over all stationary covariance kernels. I show how one might be able to perform inference that
More informationCensoring and truncation in astronomical surveys
Censoring and truncation in astronomical surveys Eric Feigelson Statistical Modeling of Cosmic Populations Penn State Summer School June 2014 Selection bias in flux-limited surveys Astronomers often struggle
More informationNonparametric Regression. Badr Missaoui
Badr Missaoui Outline Kernel and local polynomial regression. Penalized regression. We are given n pairs of observations (X 1, Y 1 ),...,(X n, Y n ) where Y i = r(x i ) + ε i, i = 1,..., n and r(x) = E(Y
More informationMaximum Smoothed Likelihood for Multivariate Nonparametric Mixtures
Maximum Smoothed Likelihood for Multivariate Nonparametric Mixtures David Hunter Pennsylvania State University, USA Joint work with: Tom Hettmansperger, Hoben Thomas, Didier Chauveau, Pierre Vandekerkhove,
More informationGAMMA-RAY LUMINOSITY AND PHOTON INDEX EVOLUTION OF FSRQ BLAZARS AND CONTRIBUTION TO THE GAMMA-RAY BACKGROUND
C 2014. The American Astronomical Society. All rights reserved. Printed in the U.S.A. doi:10.1088/0004-637x/786/2/109 GAMMA-RAY LUMINOSITY AND PHOTON INDEX EVOLUTION OF FSRQ BLAZARS AND CONTRIBUTION TO
More informationStatistics in Astronomy (II) applications. Shen, Shiyin
Statistics in Astronomy (II) applications Shen, Shiyin Contents I. Linear fitting: scaling relations I.5 correlations between parameters II. Luminosity distribution function Vmax VS maximum likelihood
More informationOptimal Ridge Detection using Coverage Risk
Optimal Ridge Detection using Coverage Risk Yen-Chi Chen Department of Statistics Carnegie Mellon University yenchic@andrew.cmu.edu Shirley Ho Department of Physics Carnegie Mellon University shirleyh@andrew.cmu.edu
More informationSTATS 200: Introduction to Statistical Inference. Lecture 29: Course review
STATS 200: Introduction to Statistical Inference Lecture 29: Course review Course review We started in Lecture 1 with a fundamental assumption: Data is a realization of a random process. The goal throughout
More informationBAYESIAN DECISION THEORY
Last updated: September 17, 2012 BAYESIAN DECISION THEORY Problems 2 The following problems from the textbook are relevant: 2.1 2.9, 2.11, 2.17 For this week, please at least solve Problem 2.3. We will
More informationCOS513: FOUNDATIONS OF PROBABILISTIC MODELS LECTURE 9: LINEAR REGRESSION
COS513: FOUNDATIONS OF PROBABILISTIC MODELS LECTURE 9: LINEAR REGRESSION SEAN GERRISH AND CHONG WANG 1. WAYS OF ORGANIZING MODELS In probabilistic modeling, there are several ways of organizing models:
More information41903: Introduction to Nonparametrics
41903: Notes 5 Introduction Nonparametrics fundamentally about fitting flexible models: want model that is flexible enough to accommodate important patterns but not so flexible it overspecializes to specific
More informationDESIGN-ADAPTIVE MINIMAX LOCAL LINEAR REGRESSION FOR LONGITUDINAL/CLUSTERED DATA
Statistica Sinica 18(2008), 515-534 DESIGN-ADAPTIVE MINIMAX LOCAL LINEAR REGRESSION FOR LONGITUDINAL/CLUSTERED DATA Kani Chen 1, Jianqing Fan 2 and Zhezhen Jin 3 1 Hong Kong University of Science and Technology,
More informationA Large-Sample Approach to Controlling the False Discovery Rate
A Large-Sample Approach to Controlling the False Discovery Rate Christopher R. Genovese Department of Statistics Carnegie Mellon University Larry Wasserman Department of Statistics Carnegie Mellon University
More informationApproximate Bayesian Computation for Astrostatistics
Approximate Bayesian Computation for Astrostatistics Jessi Cisewski Department of Statistics Yale University October 24, 2016 SAMSI Undergraduate Workshop Our goals Introduction to Bayesian methods Likelihoods,
More informationData Analysis Methods
Data Analysis Methods Successes, Opportunities, and Challenges Chad M. Schafer Department of Statistics & Data Science Carnegie Mellon University cschafer@cmu.edu The Astro/Data Science Community LSST
More informationLECTURE NOTE #3 PROF. ALAN YUILLE
LECTURE NOTE #3 PROF. ALAN YUILLE 1. Three Topics (1) Precision and Recall Curves. Receiver Operating Characteristic Curves (ROC). What to do if we do not fix the loss function? (2) The Curse of Dimensionality.
More informationDensity Estimation (II)
Density Estimation (II) Yesterday Overview & Issues Histogram Kernel estimators Ideogram Today Further development of optimization Estimating variance and bias Adaptive kernels Multivariate kernel estimation
More informationECON 721: Lecture Notes on Nonparametric Density and Regression Estimation. Petra E. Todd
ECON 721: Lecture Notes on Nonparametric Density and Regression Estimation Petra E. Todd Fall, 2014 2 Contents 1 Review of Stochastic Order Symbols 1 2 Nonparametric Density Estimation 3 2.1 Histogram
More informationMaximum Likelihood Estimation. only training data is available to design a classifier
Introduction to Pattern Recognition [ Part 5 ] Mahdi Vasighi Introduction Bayesian Decision Theory shows that we could design an optimal classifier if we knew: P( i ) : priors p(x i ) : class-conditional
More informationAnnouncements. Proposals graded
Announcements Proposals graded Kevin Jamieson 2018 1 Bayesian Methods Machine Learning CSE546 Kevin Jamieson University of Washington November 1, 2018 2018 Kevin Jamieson 2 MLE Recap - coin flips Data:
More informationA Novel Nonparametric Density Estimator
A Novel Nonparametric Density Estimator Z. I. Botev The University of Queensland Australia Abstract We present a novel nonparametric density estimator and a new data-driven bandwidth selection method with
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate
More informationAdditive Isotonic Regression
Additive Isotonic Regression Enno Mammen and Kyusang Yu 11. July 2006 INTRODUCTION: We have i.i.d. random vectors (Y 1, X 1 ),..., (Y n, X n ) with X i = (X1 i,..., X d i ) and we consider the additive
More informationGALAXY GROUPS IN THE THIRD DATA RELEASE OF THE SLOAN DIGITAL SKY SURVEY
The Astrophysical Journal, 630:759 763, 2005 September 10 # 2005. The American Astronomical Society. All rights reserved. Printed in U.S.A. GALAXY GROUPS IN THE THIRD DATA RELEASE OF THE SLOAN DIGITAL
More informationExcerpts from previous presentations. Lauren Nicholson CWRU Departments of Astronomy and Physics
Excerpts from previous presentations Lauren Nicholson CWRU Departments of Astronomy and Physics Part 1: Review of Sloan Digital Sky Survey and the Galaxy Zoo Project Part 2: Putting it all together Part
More informationInferring Galaxy Morphology Through Texture Analysis
Inferring Galaxy Morphology Through Texture Analysis 1 Galaxy Morphology Kinman Au, Christopher Genovese, Andrew Connolly Galaxies are not static objects; they evolve by interacting with the gas, dust
More informationCosmology in the Very Local Universe - Why Flow Models Matter
Cosmology in the Very Local Universe - Why Flow Models Matter Karen L. Masters Department of Astronomy, Cornell University, Ithaca, NY, 14853, USA While much of the focus of observational cosmology is
More informationLectures in AstroStatistics: Topics in Machine Learning for Astronomers
Lectures in AstroStatistics: Topics in Machine Learning for Astronomers Jessi Cisewski Yale University American Astronomical Society Meeting Wednesday, January 6, 2016 1 Statistical Learning - learning
More informationMean-Shift Tracker Computer Vision (Kris Kitani) Carnegie Mellon University
Mean-Shift Tracker 16-385 Computer Vision (Kris Kitani) Carnegie Mellon University Mean Shift Algorithm A mode seeking algorithm Fukunaga & Hostetler (1975) Mean Shift Algorithm A mode seeking algorithm
More informationData Analysis I. Dr Martin Hendry, Dept of Physics and Astronomy University of Glasgow, UK. 10 lectures, beginning October 2006
Astronomical p( y x, I) p( x, I) p ( x y, I) = p( y, I) Data Analysis I Dr Martin Hendry, Dept of Physics and Astronomy University of Glasgow, UK 10 lectures, beginning October 2006 4. Monte Carlo Methods
More informationA tailor made nonparametric density estimate
A tailor made nonparametric density estimate Daniel Carando 1, Ricardo Fraiman 2 and Pablo Groisman 1 1 Universidad de Buenos Aires 2 Universidad de San Andrés School and Workshop on Probability Theory
More informationMore on Estimation. Maximum Likelihood Estimation.
More on Estimation. In the previous chapter we looked at the properties of estimators and the criteria we could use to choose between types of estimators. Here we examine more closely some very popular
More informationDepartment of Statistics Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA citizenship: U.S.A.
Chad M. Schafer Department of Statistics Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA 15213 cschafer@cmu.edu citizenship: U.S.A. Positions Held Associate Professor Department of Statistics,
More informationPreface. 1 Nonparametric Density Estimation and Testing. 1.1 Introduction. 1.2 Univariate Density Estimation
Preface Nonparametric econometrics has become one of the most important sub-fields in modern econometrics. The primary goal of this lecture note is to introduce various nonparametric and semiparametric
More informationApproximate Bayesian Computation for the Stellar Initial Mass Function
Approximate Bayesian Computation for the Stellar Initial Mass Function Jessi Cisewski Department of Statistics Yale University SCMA6 Collaborators: Grant Weller (Savvysherpa), Chad Schafer (Carnegie Mellon),
More informationData Modeling & Analysis Techniques. Probability & Statistics. Manfred Huber
Data Modeling & Analysis Techniques Probability & Statistics Manfred Huber 2017 1 Probability and Statistics Probability and statistics are often used interchangeably but are different, related fields
More informationAutomatic Local Smoothing for Spectral Density. Abstract. This article uses local polynomial techniques to t Whittle's likelihood for spectral density
Automatic Local Smoothing for Spectral Density Estimation Jianqing Fan Department of Statistics University of North Carolina Chapel Hill, N.C. 27599-3260 Eva Kreutzberger Department of Mathematics University
More informationMarginal Specifications and a Gaussian Copula Estimation
Marginal Specifications and a Gaussian Copula Estimation Kazim Azam Abstract Multivariate analysis involving random variables of different type like count, continuous or mixture of both is frequently required
More informationIntroduction to Machine Learning
Introduction to Machine Learning 3. Instance Based Learning Alex Smola Carnegie Mellon University http://alex.smola.org/teaching/cmu2013-10-701 10-701 Outline Parzen Windows Kernels, algorithm Model selection
More informationCOMP 551 Applied Machine Learning Lecture 20: Gaussian processes
COMP 55 Applied Machine Learning Lecture 2: Gaussian processes Instructor: Ryan Lowe (ryan.lowe@cs.mcgill.ca) Slides mostly by: (herke.vanhoof@mcgill.ca) Class web page: www.cs.mcgill.ca/~hvanho2/comp55
More informationStat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
More information