Nonparametric Estimation of Luminosity Functions

Size: px
Start display at page:

Download "Nonparametric Estimation of Luminosity Functions"

Transcription

1 x x Nonparametric Estimation of Luminosity Functions Chad Schafer Department of Statistics, Carnegie Mellon University cschafer@stat.cmu.edu 1

2 Luminosity Functions The luminosity function gives the number of (quasars, galaxies, galaxy clusters,...) as a function of luminosity (L) or absolute magnitude (M) The 2-d luminosity function allows for dependence on redshift (z) Schechter Form (Schechter, 1976): ( ) ( L φ S (L) dl = n exp L ) α ( L d L L or φ S (M) dm n (α+1)(M M) exp ( 10 ) 0.4(M M) dm L ) 2

3 Schechter (1976) 3

4 From Binney and Merrifield (1998), pages : This formula was initially motivated by a simple model of galaxy formation (Press and Schecter, 1974), but it has proved to have a wider range of application than originally envisaged... With larger, deeper surveys, the limitations of the simple Schechter function start to become apparent. Particular problems on faint end, and with luminosity function evolution with redshift. Blanton, et al. (2003): Model 2-d galaxy luminosity function as Φ(M, z) = n10 0.4(z z 0)P K k=1 Φ k 2πσ 2 M exp [ 1 2 ] (M M k + (z z 0 ) Q) 2 σ 2 M 4

5 x Quasars Among the furthest observable objects Quasar PKS 2349, Observed by Hubble Telescope in 1996 Image courtesy of NASA 5

6 Absolute Magnitude Redshift Sample of 15,057 quasars from Sloan Digital Sky Survey. (Richards et al. 2006) 6

7 Luminosity Functions as Probability Density Functions Integrating a luminosity function gives the count in that bin count with 22 M 21 = φ(m) dm Proportional to the probability a randomly chosen such object is such that 22 M 21. Hence, estimating φ(m) analogous to density estimation 7

8 Density Estimation Basic Problem: Estimate f, where assume that X 1,X 2,...,X n f, i.e., for a b, P(a X i b) = b a f(x) dx Parametric Approach: Assume f belongs to some family of densities (e.g., Gaussian), and estimate the unknown parameters via maximum likelihood Nonparametric Approach: Estimate built on smoothing available data, slower rate of convergence, but less bias The histogram is a simple example of a nonparametric density estimator. 8

9 xxx Density Estimation Sample of size 303: Note that x = 20.18, s =

10 xxx Parametric Estimator density Latitude of Initial Point Assuming Gaussian, estimate µ = and σ =

11 Parametric Estimator Density Latitude of Initial Point Compared with the histogram estimate of the density. 11

12 Advantages of Parametric Form Relatively simple to fit, via maximum likelihood Functional form Easy calculation of probabilities Smaller errors, on average, provided assumption regarding density is correct 12

13 xxx Errors in Density Estimators Error at one value x: (f(x) f(x)) 2 Error accumulated over all x: ISE = (f(x) f(x)) 2 dx Mean Integrated Squared Error (MISE): ( ) MISE = E (f(x) f(x)) 2 dx 13

14 The Bias Variance Tradeoff Can write MISE = b 2 (x)dx + v(x)dx where the bias at x is and the variance at x is b(x) = E( f(x)) f(x) v(x) = V( f(x)) 14

15 The Bias Variance Tradeoff MSE Bias squared Variance Less Optimal smoothing More 15

16 Histograms Create bins B 1,B 2,...,B m of width h. Define f n (x) = m j=1 p j h I(x B j). where p j is proportion of observations in B j. Note that f n (x)dx = 1. A histogram is an example of a nonparametric density estimator. Note that it is controlled by the tuning parameter h, the bin width. 16

17 Histograms Density Latitude of Initial Point Seems like too many bins, i.e. h is too small. 17

18 Histograms Density Latitude of Initial Point Seems like too few bins, i.e. h is too large. 18

19 xxx Histograms Latitude of Initial Point Density Seems better. 19

20 Histograms Density Latitude of Initial Point Clear that normal curve is not a good fit. 20

21 Histograms For histograms, ( MISE = E ) (f(x) f(x)) 2 dx h2 12 (f (u)) 2 du + 1 nh The value h that minimizes this is h = 1 ( 1/3 6 n 1/3 (f. (u)) du) 2 and then MISE C 2 n 2/3 21

22 Kernel Density Estimation We can improve histograms by introducing smoothing. f(x) = 1 n ( ) x Xi K. nh h Here, K( ) is the kernel function, itself a smooth density. i=1 22

23 Kernels A kernel is any smooth function K such that K(x) 0 and K(x)dx = 1, xk(x)dx = 0 and σk 2 x 2 K(x)dx > 0. Some commonly used kernels are the following: the boxcar kernel : K(x) = 1 I( x < 1), 2 the Gaussian kernel : K(x) = 1 e x2 /2, 2π the Epanechnikov kernel : K(x) = 3 4 (1 x2 )I( x < 1) the tricube kernel : K(x) = (1 x 3 ) 3 I( x < 1). 23

24 xxx Kernels

25 xxx Kernel Density Estimation Places a smoothed-out lump of mass of size 1/n over each data point The shape of the lumps is controlled by K( ); their width is controlled by h. 25

26 Kernel Density Estimation Density Latitude of Initial Point h chosen too large. 26

27 xxx Kernel Density Estimation Density Latitude of Initial Point h chosen too small. 27

28 Kernel Density Estimation Density Latitude of Initial Point Just right. 28

29 xxx Kernel Density Estimation MSE 1 4 c2 1h 4 (f (x)) 2 dx + K 2 (x)dx nh Optimal bandwidth is h = ( c2 c 2 1A(f)n ) 1/5 where c 1 = x 2 K(x)dx, c 2 = K(x) 2 dx and A(f) = (f (x)) 2 dx. Then, MSE C 3 n 4/5. 29

30 The Bias Variance Tradeoff For many smoothers: which is minimized at MISE c 1 h 4 + c 2 nh h = O ( 1 n 1/5 ) Hence, MISE = O whereas, for parametric problems MISE = O ( 1 n 4/5 ( ) 1 n ) 30

31 xxx Comparisons So, for parametric form, if it s correct, MISE C 1 n For histograms, MISE C 2 n 2/3 For estimators based on smoothing, MISE C 3 n 4/5 31

32 xxx Comparisons f(x) truth closest normal x Truth is standard normal. 32

33 Comparisons MSE 0e+00 1e 05 2e 05 3e 05 4e sample size (n) normal kernel Assuming normality has its advantages. 33

34 Comparisons truth closest normal f(x) Truth is t-distribution with 5 degrees of freedom. x 34

35 Comparisons normal kernel MSE 5.0e e e sample size (n) Normal assumption costs, even for moderate sample size. 35

36 Cross-Validation Objective, data-driven choice of h is possible: ( ) MISE = E (f(x) f(x)) 2 dx ( = E = J(h) + ) f(x) 2 dx f(x) 2 dx ( 2E ) f(x)f(x)dx + f(x) 2 dx Unbiased Estimator of J(h): Ĵ(h) = ( ) 2 2 fn (x) dx n n i=1 f ( i) (X i ) where f ( i) is the density estimator obtained after removing the i th observation. 36

37 x Local Density Estimation The idea of local modelling (recal local polynomial regression) extends to density estimation. The local likelihood density estimator developed independently by Loader (1996) and Hjort and Jones (1996) allows for fitting of nonparametric models of this type. 37

38 Local Density Estimation Model assumes that for u near x, log(f(x)) a 0 + a 1 (u x) + + a p (u x) p Thus, ( ) log f(x) = â 0 + â 1 (u x) + + â p (u x) p is the local (near x) estimate of log( f( )) Estimates obtained by maximizing kernel-weigthed likelihood Repeated for a grid of values x x x x 38

39 Log Density A few local linear estimates of log( f( )) 39

40 Log Density Smoothed toghether to get log( f( )) 40

41 Back to Luminosity Functions... Simple parametric forms not realistic for large data sets Could build more complex parametric forms, e.g., Blanton, et al. (2003) Φ(M, z) = n10 0.4(z z 0)P K k=1 Φ k 2πσ 2 M exp [ 1 2 ] (M M k + (z z 0 ) Q) 2 σ 2 M Nonparametric approaches need to account for observational limitations, distinguish physical variations from observational variations 41

42 Truncation Suppose that X 1,X 2,...,X m f, but data are truncated: only get to see those X i such that X i k. Formally, we get to observe X 1,X 2,...,X n f, where f(x) f R k (x) = f(x)dx, if x k 0, if x > k Clearly, must take truncation into account when using the observed data to estimate f. 42

43 Absolute Magnitude Redshift Sample of 15,057 quasars from Sloan Digital Sky Survey. (Richards et al. 2006) 43

44 1/V max Estimator (Schmidt, 1968; Eales, 1993) Observe objects of absolute magnitudes M 1,M 2,...,M n Divide absolute magnitude range into bins. Estimate φ over j th bin as bin j φ(m) = i bin j 1 V max (i) where V max (i) is the volume over which an object of magnitude M i can be detected by the survey This is a weighted histogram 44

45 Nonparametric Maximum Likelihood (NPMLE) Lynden-Bell (1971): Consider the case of one-sided truncation Efron and Petrosian (1999): Two-sided (double) truncation Know that NPMLE will assign all probability to oberved values, but they will not be equal probabilities NPMLE estimators for this case are implemented in R package DTDA. 45

46 NPMLE Examples Data Value Observation Number Vertical lines are truncation ranges, dots are data values. 46

47 NPMLE Examples Data Value Observation Number The probability assigned to each by the NPMLE. 47

48 NPMLE Examples Data Value Observation Number Differential truncation. 48

49 NPMLE Examples Data Value Observation Number Both of the intervals (, 2) and (2, ) has probability

50 NPMLE Examples Data Value Observation Number An example where it performs poorly. 50

51 NPMLE Examples Data Value Observation Number xxx 51

52 NPMLE Examples Data Value Observation Number Example from Efron and Petrosian (1999) 52

53 NPMLE Examples Data Value Observation Number 53

54 Efron and Petrosian (1999) Quasar Example 54

55 Efron and Petrosian (1999) Quasar Example 55

56 Efron and Petrosian (1999) Quasar Example The SEF fit is a special exponential family model that assumes that log f(x) = β 0 + β 1 x + β 2 x 2 + β 3 x 3 This model can be fit via maximum likelihood. BUT: Each of these fits assumes independence between redshift and luminosity, i.e., no evolution of the luminosity function with redshift. 56

57 Absolute Magnitude Redshift Sample of 15,057 quasars from Sloan Digital Sky Survey. (Richards et al. 2006) 57

58 The Challenge: Model selection Goal: Given the truncated data shown in the scatter plot, estimate the bivariate density (φ(z,m)). Too Strong: Assume independence: φ(z, M) = f(z)g(m), or assume parametric form: φ(z,m,θ) Too Weak: Use fully nonparametric approach 58

59 xxx A Semiparametric Model Schafer (2007) We utilize a semiparametric model which allows one to place minimal assumptions on the form of the bivariate density and greatly diminish bias due to truncation and boundary effects. Decompose the bivariate density φ(z,m) into log(φ(z,m)) = f(z) + g(m) + h(z,m,θ) where f(z) and g(m) are estimated using nonparametric methods and h(z,m,θ) will take an assumed parametric form; it is intended to model the dependence between the two random variables. We use h(z,m,θ) = θzm, but another choice could be made, e.g. motivated by a physical theory. 59

60 Fitting the model: The motivating (naive) idea Assume independence and write φ(z, M) = f(z)g(m) where f( ) is the density for redshift and g( ) is the density for absolute magnitude. x Absolute Magnitude A A(4.0,.) x Let A denote the region outside of which of the data are truncated and let A(z, ) denote the cross-section of A at z. Assume, for the moment, that g( ) is known Redshift 60

61 x The available data allow for estimation of f (z) φ(z,m)dm = f(z) A(z, ) A(z, ) g(m)dm for all z, since it is the density of the observable redshift. Assuming for the moment that g( ) were known, it is possible to estimate f( ) using f(z) = f / ( ) (z) g(m)dm. A(z, ) Starting with an initial guess at g( ), we could iterate between assuming g( ) is known, and estimating f( ), and vice versa. But, the variance of this estimator could be huge: Consider the behavior of f(z) for z where A(z, ) g(m)dm is small. 61

62 Absolute Magnitude Redshift Depicting A(1.5, ) 62

63 Density % Unobserved 24% Observed Absolute Magnitude 24% of area under assumed g( ) falls in A(1.5, ) 63

64 Percent Observed Redshift (z) Percent of area under g( ) in A(z, ), as a function of z. 64

65 Estimated Redshift Density Redshift Smooth f ( ) does not lead to smooth f( ) 65

66 x Local density estimation Instead, we write ( log(f (z)) = log(f(z)) + log A(z, ) ) g(m)dm and treat ( log A(z, ) ) g(m)dm as an offset term in a model for log(f (z)) and consider only forms for f( ) which are controlled by smoothing parameters chosen by the user. The local likelihood density estimator developed independently by Loader (1996) and Hjort and Jones (1996) allows for fitting of nonparametric models of this type; the offset is easily incorporated. 66

67 x M-estimators This formulation allows the estimation problem to be couched as an M-estimator, i.e. choose parameters of the local models to maximize criterion n ψ(z i,m i ) i=1 Leads to estimates of standard errors, etc. The parametric component h(z,m,θ) is also easily included in the offset. Can use weights allows for varying selection function: n w i ψ(z i,m i ) i=1 67

68 Bandwidth Selection via Cross-Validation One ideal choice of the smoothing parameters would minimize the integrated mean squared error: [ ) ] 2 E ( φ(z,m) φ(z,m) dz dm. A An unbiased estimator of the integrated mean squared error is φ 2 (z,m)dz dm 2 n φ ( j) (z j,m j ) n A where φ ( j) (z j,m j ) denotes the estimate found when omitting the j th data pair from the analysis efficiently estimated here. j=1 68

69 Absolute Magnitude Redshift Sample of 15,057 quasars from the Sloan Digital Sky Survey. 69

70 Absolute Magnitude Redshift Same plot, with dots scaled by value of selection function for that observation. 70

71 Key aspects of the approach Allows for the removal of the independence assumption, without assuming a parametric form Iterative algorithm not computationally burdensome; must converge to a unique estimate Statistical theory leads to good approximations to standard errors for all estimated quantities Cross-validation estimator of integrated mean squared error is easily calculated, leading to good bandwidth selection Natural incorporation of the selection function 71

72 LSCV λ z λ M = 0.15 λ M = 0.16 λ M = 0.17 λ M = 0.18 λ M = 0.19 Finding the optimal bandwidths. 72

73 Absolute Magnitude Redshift Estimate of bivariate density, i.e. 2-D luminosity function. 73

74 log(ρ(z))(m < )(Mpc 3 ) Redshift Marginal density for redshift. 74

75 Φ(M i ) (Mpc 3 mag 1 ) z = 0.49 z = 1.25 z = 2.01 Φ(M i ) (Mpc 3 mag 1 ) z = 2.8 z = 3.75 z = M i M i M i Slices of the bivariate estimate, compared with Richards, et al. (2006). 75

76 z = 1 z = Φ(M) (Mpc 3 mag 1 ) Φ(M) (Mpc 3 mag 1 ) z = 3 z = M M Estimates based on simulated data. 76

77 Nonparametric Bayesian Approach Kelly et al. (2008): Model 2-d luminosity function as mixture of Gaussians. Parameterized form allows for Bayesian inference Natural uncertainty measures for any functions of the luminosity function 77

78 Figure 3(a) from Kelly, et al. (2008) 78

79 Conclusion Luminosity functions as probability densities The potential of nonparametric density estimation Moving beyond the independence assumption and binning 79

80 References Binney and Merrifield (1998). Galactic Astronomy. Princeton University Press. Blanton (2003). ApJ. 592, Eales (1993). ApJ Efron and Petrosian (1999). JASA Kelly, et al. (2008). ApJ Lynden-Bell (1971). MNRAS Press and Schechter (1974). ApJ 187, Richards et al. (2006). AJ. 131, Schafer (2007). ApJ 661, Schechter (1976). ApJ (203), Schmidt (1968). ApJ (151), Takeuchi, et al. (2000). ApJS (129),

81 Acknowledgements Peter Freeman, Chris Genovese, and Larry Wasserman, Department of Statistics, CMU. Sloan Digital Sky Survey Research supported by NSF grants # and # For more information... See cschafer/bivtrunc Contact Chad at 81

Nonparametric Statistics

Nonparametric Statistics Nonparametric Statistics Jessi Cisewski Yale University Astrostatistics Summer School - XI Wednesday, June 3, 2015 1 Overview Many of the standard statistical inference procedures are based on assumptions

More information

arxiv:astro-ph/ v1 15 Feb 2007

arxiv:astro-ph/ v1 15 Feb 2007 A Statistical Method for Estimating Luminosity Functions using Truncated Data Chad M. Schafer cschafer@stat.cmu.edu arxiv:astro-ph/0702401v1 15 Feb 2007 Department of Statistics, Carnegie Mellon University

More information

Improved Astronomical Inferences via Nonparametric Density Estimation

Improved Astronomical Inferences via Nonparametric Density Estimation Improved Astronomical Inferences via Nonparametric Density Estimation Chad M. Schafer, InCA Group www.incagroup.org Department of Statistics Carnegie Mellon University Work Supported by NSF, NASA-AISR

More information

Introduction to Regression

Introduction to Regression Introduction to Regression p. 1/97 Introduction to Regression Chad Schafer cschafer@stat.cmu.edu Carnegie Mellon University Introduction to Regression p. 1/97 Acknowledgement Larry Wasserman, All of Nonparametric

More information

Introduction to Regression

Introduction to Regression Introduction to Regression Chad M. Schafer cschafer@stat.cmu.edu Carnegie Mellon University Introduction to Regression p. 1/100 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression

More information

Introduction to Regression

Introduction to Regression Introduction to Regression David E Jones (slides mostly by Chad M Schafer) June 1, 2016 1 / 102 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression Nonparametric Procedures

More information

Introduction to Regression

Introduction to Regression Introduction to Regression Chad M. Schafer May 20, 2015 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression Nonparametric Procedures Cross Validation Local Polynomial Regression

More information

Addressing the Challenges of Luminosity Function Estimation via Likelihood-Free Inference

Addressing the Challenges of Luminosity Function Estimation via Likelihood-Free Inference Addressing the Challenges of Luminosity Function Estimation via Likelihood-Free Inference Chad M. Schafer www.stat.cmu.edu/ cschafer Department of Statistics Carnegie Mellon University June 2011 1 Acknowledgements

More information

Statistics: Learning models from data

Statistics: Learning models from data DS-GA 1002 Lecture notes 5 October 19, 2015 Statistics: Learning models from data Learning models from data that are assumed to be generated probabilistically from a certain unknown distribution is a crucial

More information

Galaxy Morphology Refresher. Driver et al. 2006

Galaxy Morphology Refresher. Driver et al. 2006 Galaxy Morphology Refresher Driver et al. 2006 http://www.galaxyzoo.org Galaxy Luminosity Functions Luminosity Function is the number density (# per volume) of some population of objects (e.g., galaxies)

More information

group. redshifts. However, the sampling is fairly complete out to about z = 0.2.

group. redshifts. However, the sampling is fairly complete out to about z = 0.2. Non parametric Inference in Astrophysics Larry Wasserman, Chris Miller, Bob Nichol, Chris Genovese, Woncheol Jang, Andy Connolly, Andrew Moore, Jeff Schneider and the PICA group. 1 We discuss non parametric

More information

Quantitative Economics for the Evaluation of the European Policy. Dipartimento di Economia e Management

Quantitative Economics for the Evaluation of the European Policy. Dipartimento di Economia e Management Quantitative Economics for the Evaluation of the European Policy Dipartimento di Economia e Management Irene Brunetti 1 Davide Fiaschi 2 Angela Parenti 3 9 ottobre 2015 1 ireneb@ec.unipi.it. 2 davide.fiaschi@unipi.it.

More information

Space Telescope Science Institute statistics mini-course. October Inference I: Estimation, Confidence Intervals, and Tests of Hypotheses

Space Telescope Science Institute statistics mini-course. October Inference I: Estimation, Confidence Intervals, and Tests of Hypotheses Space Telescope Science Institute statistics mini-course October 2011 Inference I: Estimation, Confidence Intervals, and Tests of Hypotheses James L Rosenberger Acknowledgements: Donald Richards, William

More information

12 - Nonparametric Density Estimation

12 - Nonparametric Density Estimation ST 697 Fall 2017 1/49 12 - Nonparametric Density Estimation ST 697 Fall 2017 University of Alabama Density Review ST 697 Fall 2017 2/49 Continuous Random Variables ST 697 Fall 2017 3/49 1.0 0.8 F(x) 0.6

More information

Chapter 9. Non-Parametric Density Function Estimation

Chapter 9. Non-Parametric Density Function Estimation 9-1 Density Estimation Version 1.1 Chapter 9 Non-Parametric Density Function Estimation 9.1. Introduction We have discussed several estimation techniques: method of moments, maximum likelihood, and least

More information

Nonparametric Methods

Nonparametric Methods Nonparametric Methods Michael R. Roberts Department of Finance The Wharton School University of Pennsylvania July 28, 2009 Michael R. Roberts Nonparametric Methods 1/42 Overview Great for data analysis

More information

O Combining cross-validation and plug-in methods - for kernel density bandwidth selection O

O Combining cross-validation and plug-in methods - for kernel density bandwidth selection O O Combining cross-validation and plug-in methods - for kernel density selection O Carlos Tenreiro CMUC and DMUC, University of Coimbra PhD Program UC UP February 18, 2011 1 Overview The nonparametric problem

More information

Econ 582 Nonparametric Regression

Econ 582 Nonparametric Regression Econ 582 Nonparametric Regression Eric Zivot May 28, 2013 Nonparametric Regression Sofarwehaveonlyconsideredlinearregressionmodels = x 0 β + [ x ]=0 [ x = x] =x 0 β = [ x = x] [ x = x] x = β The assume

More information

Censoring and truncation in astronomical surveys. Eric Feigelson (Astronomy & Astrophysics, Penn State)

Censoring and truncation in astronomical surveys. Eric Feigelson (Astronomy & Astrophysics, Penn State) Censoring and truncation in astronomical surveys Eric Feigelson (Astronomy & Astrophysics, Penn State) Outline 1. Examples of astronomical surveys 2. One survey: Truncation and spatial/luminosity distributions

More information

Measurement Error and Linear Regression of Astronomical Data. Brandon Kelly Penn State Summer School in Astrostatistics, June 2007

Measurement Error and Linear Regression of Astronomical Data. Brandon Kelly Penn State Summer School in Astrostatistics, June 2007 Measurement Error and Linear Regression of Astronomical Data Brandon Kelly Penn State Summer School in Astrostatistics, June 2007 Classical Regression Model Collect n data points, denote i th pair as (η

More information

Chapter 9. Non-Parametric Density Function Estimation

Chapter 9. Non-Parametric Density Function Estimation 9-1 Density Estimation Version 1.2 Chapter 9 Non-Parametric Density Function Estimation 9.1. Introduction We have discussed several estimation techniques: method of moments, maximum likelihood, and least

More information

Histogram Härdle, Müller, Sperlich, Werwatz, 1995, Nonparametric and Semiparametric Models, An Introduction

Histogram Härdle, Müller, Sperlich, Werwatz, 1995, Nonparametric and Semiparametric Models, An Introduction Härdle, Müller, Sperlich, Werwatz, 1995, Nonparametric and Semiparametric Models, An Introduction Tine Buch-Kromann Construction X 1,..., X n iid r.v. with (unknown) density, f. Aim: Estimate the density

More information

Kernel density estimation with doubly truncated data

Kernel density estimation with doubly truncated data Kernel density estimation with doubly truncated data Jacobo de Uña-Álvarez jacobo@uvigo.es http://webs.uvigo.es/jacobo (joint with Carla Moreira) Department of Statistics and OR University of Vigo Spain

More information

Density estimation Nonparametric conditional mean estimation Semiparametric conditional mean estimation. Nonparametrics. Gabriel Montes-Rojas

Density estimation Nonparametric conditional mean estimation Semiparametric conditional mean estimation. Nonparametrics. Gabriel Montes-Rojas 0 0 5 Motivation: Regression discontinuity (Angrist&Pischke) Outcome.5 1 1.5 A. Linear E[Y 0i X i] 0.2.4.6.8 1 X Outcome.5 1 1.5 B. Nonlinear E[Y 0i X i] i 0.2.4.6.8 1 X utcome.5 1 1.5 C. Nonlinearity

More information

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS Parametric Distributions Basic building blocks: Need to determine given Representation: or? Recall Curve Fitting Binary Variables

More information

Nonparametric Inference in Cosmology and Astrophysics: Biases and Variants

Nonparametric Inference in Cosmology and Astrophysics: Biases and Variants Nonparametric Inference in Cosmology and Astrophysics: Biases and Variants Christopher R. Genovese Department of Statistics Carnegie Mellon University http://www.stat.cmu.edu/ ~ genovese/ Collaborators:

More information

Sparse Nonparametric Density Estimation in High Dimensions Using the Rodeo

Sparse Nonparametric Density Estimation in High Dimensions Using the Rodeo Sparse Nonparametric Density Estimation in High Dimensions Using the Rodeo Han Liu John Lafferty Larry Wasserman Statistics Department Computer Science Department Machine Learning Department Carnegie Mellon

More information

Minimum Hellinger Distance Estimation in a. Semiparametric Mixture Model

Minimum Hellinger Distance Estimation in a. Semiparametric Mixture Model Minimum Hellinger Distance Estimation in a Semiparametric Mixture Model Sijia Xiang 1, Weixin Yao 1, and Jingjing Wu 2 1 Department of Statistics, Kansas State University, Manhattan, Kansas, USA 66506-0802.

More information

Intensity Analysis of Spatial Point Patterns Geog 210C Introduction to Spatial Data Analysis

Intensity Analysis of Spatial Point Patterns Geog 210C Introduction to Spatial Data Analysis Intensity Analysis of Spatial Point Patterns Geog 210C Introduction to Spatial Data Analysis Chris Funk Lecture 4 Spatial Point Patterns Definition Set of point locations with recorded events" within study

More information

Confidence intervals for kernel density estimation

Confidence intervals for kernel density estimation Stata User Group - 9th UK meeting - 19/20 May 2003 Confidence intervals for kernel density estimation Carlo Fiorio c.fiorio@lse.ac.uk London School of Economics and STICERD Stata User Group - 9th UK meeting

More information

Log-Density Estimation with Application to Approximate Likelihood Inference

Log-Density Estimation with Application to Approximate Likelihood Inference Log-Density Estimation with Application to Approximate Likelihood Inference Martin Hazelton 1 Institute of Fundamental Sciences Massey University 19 November 2015 1 Email: m.hazelton@massey.ac.nz WWPMS,

More information

Discrete Mathematics and Probability Theory Fall 2015 Lecture 21

Discrete Mathematics and Probability Theory Fall 2015 Lecture 21 CS 70 Discrete Mathematics and Probability Theory Fall 205 Lecture 2 Inference In this note we revisit the problem of inference: Given some data or observations from the world, what can we infer about

More information

Bayesian estimation of the discrepancy with misspecified parametric models

Bayesian estimation of the discrepancy with misspecified parametric models Bayesian estimation of the discrepancy with misspecified parametric models Pierpaolo De Blasi University of Torino & Collegio Carlo Alberto Bayesian Nonparametrics workshop ICERM, 17-21 September 2012

More information

Dr Martin Hendry Dept of Physics and Astronomy University of Glasgow, UK

Dr Martin Hendry Dept of Physics and Astronomy University of Glasgow, UK Dr Martin Hendry Dept of Physics and Astronomy University of Glasgow, UK martin@astroglaacuk Dr Martin Hendry, Russell Johnston Dept of Physics and Astronomy University of Glasgow, UK Who am I? William

More information

Motivational Example

Motivational Example Motivational Example Data: Observational longitudinal study of obesity from birth to adulthood. Overall Goal: Build age-, gender-, height-specific growth charts (under 3 year) to diagnose growth abnomalities.

More information

Pattern Recognition and Machine Learning. Bishop Chapter 2: Probability Distributions

Pattern Recognition and Machine Learning. Bishop Chapter 2: Probability Distributions Pattern Recognition and Machine Learning Chapter 2: Probability Distributions Cécile Amblard Alex Kläser Jakob Verbeek October 11, 27 Probability Distributions: General Density Estimation: given a finite

More information

Gaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012

Gaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012 Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature

More information

Statistics notes. A clear statistical framework formulates the logic of what we are doing and why. It allows us to make precise statements.

Statistics notes. A clear statistical framework formulates the logic of what we are doing and why. It allows us to make precise statements. Statistics notes Introductory comments These notes provide a summary or cheat sheet covering some basic statistical recipes and methods. These will be discussed in more detail in the lectures! What is

More information

Exploiting Sparse Non-Linear Structure in Astronomical Data

Exploiting Sparse Non-Linear Structure in Astronomical Data Exploiting Sparse Non-Linear Structure in Astronomical Data Ann B. Lee Department of Statistics and Department of Machine Learning, Carnegie Mellon University Joint work with P. Freeman, C. Schafer, and

More information

Boundary Correction Methods in Kernel Density Estimation Tom Alberts C o u(r)a n (t) Institute joint work with R.J. Karunamuni University of Alberta

Boundary Correction Methods in Kernel Density Estimation Tom Alberts C o u(r)a n (t) Institute joint work with R.J. Karunamuni University of Alberta Boundary Correction Methods in Kernel Density Estimation Tom Alberts C o u(r)a n (t) Institute joint work with R.J. Karunamuni University of Alberta November 29, 2007 Outline Overview of Kernel Density

More information

Nonparametric Inference and the Dark Energy Equation of State

Nonparametric Inference and the Dark Energy Equation of State Nonparametric Inference and the Dark Energy Equation of State Christopher R. Genovese Peter E. Freeman Larry Wasserman Department of Statistics Carnegie Mellon University http://www.stat.cmu.edu/ ~ genovese/

More information

Galaxy number counts in a presence of the graviton background

Galaxy number counts in a presence of the graviton background arxiv:astro-ph/0606223v3 15 Jun 2006 Galaxy number counts in a presence of the graviton background Michael A. Ivanov Physics Dept., Belarus State University of Informatics and Radioelectronics, 6 P. Brovka

More information

Nonparametric Density Estimation

Nonparametric Density Estimation Nonparametric Density Estimation Econ 690 Purdue University Justin L. Tobias (Purdue) Nonparametric Density Estimation 1 / 29 Density Estimation Suppose that you had some data, say on wages, and you wanted

More information

Akaike Information Criterion to Select the Parametric Detection Function for Kernel Estimator Using Line Transect Data

Akaike Information Criterion to Select the Parametric Detection Function for Kernel Estimator Using Line Transect Data Journal of Modern Applied Statistical Methods Volume 12 Issue 2 Article 21 11-1-2013 Akaike Information Criterion to Select the Parametric Detection Function for Kernel Estimator Using Line Transect Data

More information

Bayes via forward simulation. Approximate Bayesian Computation

Bayes via forward simulation. Approximate Bayesian Computation : Approximate Bayesian Computation Jessi Cisewski Carnegie Mellon University June 2014 What is Approximate Bayesian Computing? Likelihood-free approach Works by simulating from the forward process What

More information

Time Series and Forecasting Lecture 4 NonLinear Time Series

Time Series and Forecasting Lecture 4 NonLinear Time Series Time Series and Forecasting Lecture 4 NonLinear Time Series Bruce E. Hansen Summer School in Economics and Econometrics University of Crete July 23-27, 2012 Bruce Hansen (University of Wisconsin) Foundations

More information

Adaptive Nonparametric Density Estimators

Adaptive Nonparametric Density Estimators Adaptive Nonparametric Density Estimators by Alan J. Izenman Introduction Theoretical results and practical application of histograms as density estimators usually assume a fixed-partition approach, where

More information

Sparse Nonparametric Density Estimation in High Dimensions Using the Rodeo

Sparse Nonparametric Density Estimation in High Dimensions Using the Rodeo Outline in High Dimensions Using the Rodeo Han Liu 1,2 John Lafferty 2,3 Larry Wasserman 1,2 1 Statistics Department, 2 Machine Learning Department, 3 Computer Science Department, Carnegie Mellon University

More information

The flickering luminosity method

The flickering luminosity method The flickering luminosity method Martin Feix in collaboration with Adi Nusser (Technion) and Enzo Branchini (Roma Tre) Institut d Astrophysique de Paris SSG16 Workshop, February 2nd 2016 Outline 1 Motivation

More information

Are Bayesian Inferences Weak for Wasserman s Example?

Are Bayesian Inferences Weak for Wasserman s Example? Are Bayesian Inferences Weak for Wasserman s Example? Longhai Li longhai@math.usask.ca Department of Mathematics and Statistics University of Saskatchewan Saskatoon, Saskatchewan, S7N 5E6 Canada Quebec

More information

Introduction to machine learning and pattern recognition Lecture 2 Coryn Bailer-Jones

Introduction to machine learning and pattern recognition Lecture 2 Coryn Bailer-Jones Introduction to machine learning and pattern recognition Lecture 2 Coryn Bailer-Jones http://www.mpia.de/homes/calj/mlpr_mpia2008.html 1 1 Last week... supervised and unsupervised methods need adaptive

More information

Galaxy Luminosity Function

Galaxy Luminosity Function Galaxy Luminosity Function Methods: Vmax Max Likelihood! Observations of LF: Shape of LF Field LF LF in Groups and Clusters Luminosity Function Φ(M)dM is the number of galaxies per unit volume with absolute

More information

Estimation of a Two-component Mixture Model

Estimation of a Two-component Mixture Model Estimation of a Two-component Mixture Model Bodhisattva Sen 1,2 University of Cambridge, Cambridge, UK Columbia University, New York, USA Indian Statistical Institute, Kolkata, India 6 August, 2012 1 Joint

More information

Curve Fitting Re-visited, Bishop1.2.5

Curve Fitting Re-visited, Bishop1.2.5 Curve Fitting Re-visited, Bishop1.2.5 Maximum Likelihood Bishop 1.2.5 Model Likelihood differentiation p(t x, w, β) = Maximum Likelihood N N ( t n y(x n, w), β 1). (1.61) n=1 As we did in the case of the

More information

Basic Sampling Methods

Basic Sampling Methods Basic Sampling Methods Sargur Srihari srihari@cedar.buffalo.edu 1 1. Motivation Topics Intractability in ML How sampling can help 2. Ancestral Sampling Using BNs 3. Transforming a Uniform Distribution

More information

Kernel density estimation

Kernel density estimation Kernel density estimation Patrick Breheny October 18 Patrick Breheny STA 621: Nonparametric Statistics 1/34 Introduction Kernel Density Estimation We ve looked at one method for estimating density: histograms

More information

Cheng Soon Ong & Christian Walder. Canberra February June 2018

Cheng Soon Ong & Christian Walder. Canberra February June 2018 Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 2018 Outlines Overview Introduction Linear Algebra Probability Linear Regression

More information

Local Polynomial Modelling and Its Applications

Local Polynomial Modelling and Its Applications Local Polynomial Modelling and Its Applications J. Fan Department of Statistics University of North Carolina Chapel Hill, USA and I. Gijbels Institute of Statistics Catholic University oflouvain Louvain-la-Neuve,

More information

Density Estimation. Seungjin Choi

Density Estimation. Seungjin Choi Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/

More information

A Process over all Stationary Covariance Kernels

A Process over all Stationary Covariance Kernels A Process over all Stationary Covariance Kernels Andrew Gordon Wilson June 9, 0 Abstract I define a process over all stationary covariance kernels. I show how one might be able to perform inference that

More information

Censoring and truncation in astronomical surveys

Censoring and truncation in astronomical surveys Censoring and truncation in astronomical surveys Eric Feigelson Statistical Modeling of Cosmic Populations Penn State Summer School June 2014 Selection bias in flux-limited surveys Astronomers often struggle

More information

Nonparametric Regression. Badr Missaoui

Nonparametric Regression. Badr Missaoui Badr Missaoui Outline Kernel and local polynomial regression. Penalized regression. We are given n pairs of observations (X 1, Y 1 ),...,(X n, Y n ) where Y i = r(x i ) + ε i, i = 1,..., n and r(x) = E(Y

More information

Maximum Smoothed Likelihood for Multivariate Nonparametric Mixtures

Maximum Smoothed Likelihood for Multivariate Nonparametric Mixtures Maximum Smoothed Likelihood for Multivariate Nonparametric Mixtures David Hunter Pennsylvania State University, USA Joint work with: Tom Hettmansperger, Hoben Thomas, Didier Chauveau, Pierre Vandekerkhove,

More information

GAMMA-RAY LUMINOSITY AND PHOTON INDEX EVOLUTION OF FSRQ BLAZARS AND CONTRIBUTION TO THE GAMMA-RAY BACKGROUND

GAMMA-RAY LUMINOSITY AND PHOTON INDEX EVOLUTION OF FSRQ BLAZARS AND CONTRIBUTION TO THE GAMMA-RAY BACKGROUND C 2014. The American Astronomical Society. All rights reserved. Printed in the U.S.A. doi:10.1088/0004-637x/786/2/109 GAMMA-RAY LUMINOSITY AND PHOTON INDEX EVOLUTION OF FSRQ BLAZARS AND CONTRIBUTION TO

More information

Statistics in Astronomy (II) applications. Shen, Shiyin

Statistics in Astronomy (II) applications. Shen, Shiyin Statistics in Astronomy (II) applications Shen, Shiyin Contents I. Linear fitting: scaling relations I.5 correlations between parameters II. Luminosity distribution function Vmax VS maximum likelihood

More information

Optimal Ridge Detection using Coverage Risk

Optimal Ridge Detection using Coverage Risk Optimal Ridge Detection using Coverage Risk Yen-Chi Chen Department of Statistics Carnegie Mellon University yenchic@andrew.cmu.edu Shirley Ho Department of Physics Carnegie Mellon University shirleyh@andrew.cmu.edu

More information

STATS 200: Introduction to Statistical Inference. Lecture 29: Course review

STATS 200: Introduction to Statistical Inference. Lecture 29: Course review STATS 200: Introduction to Statistical Inference Lecture 29: Course review Course review We started in Lecture 1 with a fundamental assumption: Data is a realization of a random process. The goal throughout

More information

BAYESIAN DECISION THEORY

BAYESIAN DECISION THEORY Last updated: September 17, 2012 BAYESIAN DECISION THEORY Problems 2 The following problems from the textbook are relevant: 2.1 2.9, 2.11, 2.17 For this week, please at least solve Problem 2.3. We will

More information

COS513: FOUNDATIONS OF PROBABILISTIC MODELS LECTURE 9: LINEAR REGRESSION

COS513: FOUNDATIONS OF PROBABILISTIC MODELS LECTURE 9: LINEAR REGRESSION COS513: FOUNDATIONS OF PROBABILISTIC MODELS LECTURE 9: LINEAR REGRESSION SEAN GERRISH AND CHONG WANG 1. WAYS OF ORGANIZING MODELS In probabilistic modeling, there are several ways of organizing models:

More information

41903: Introduction to Nonparametrics

41903: Introduction to Nonparametrics 41903: Notes 5 Introduction Nonparametrics fundamentally about fitting flexible models: want model that is flexible enough to accommodate important patterns but not so flexible it overspecializes to specific

More information

DESIGN-ADAPTIVE MINIMAX LOCAL LINEAR REGRESSION FOR LONGITUDINAL/CLUSTERED DATA

DESIGN-ADAPTIVE MINIMAX LOCAL LINEAR REGRESSION FOR LONGITUDINAL/CLUSTERED DATA Statistica Sinica 18(2008), 515-534 DESIGN-ADAPTIVE MINIMAX LOCAL LINEAR REGRESSION FOR LONGITUDINAL/CLUSTERED DATA Kani Chen 1, Jianqing Fan 2 and Zhezhen Jin 3 1 Hong Kong University of Science and Technology,

More information

A Large-Sample Approach to Controlling the False Discovery Rate

A Large-Sample Approach to Controlling the False Discovery Rate A Large-Sample Approach to Controlling the False Discovery Rate Christopher R. Genovese Department of Statistics Carnegie Mellon University Larry Wasserman Department of Statistics Carnegie Mellon University

More information

Approximate Bayesian Computation for Astrostatistics

Approximate Bayesian Computation for Astrostatistics Approximate Bayesian Computation for Astrostatistics Jessi Cisewski Department of Statistics Yale University October 24, 2016 SAMSI Undergraduate Workshop Our goals Introduction to Bayesian methods Likelihoods,

More information

Data Analysis Methods

Data Analysis Methods Data Analysis Methods Successes, Opportunities, and Challenges Chad M. Schafer Department of Statistics & Data Science Carnegie Mellon University cschafer@cmu.edu The Astro/Data Science Community LSST

More information

LECTURE NOTE #3 PROF. ALAN YUILLE

LECTURE NOTE #3 PROF. ALAN YUILLE LECTURE NOTE #3 PROF. ALAN YUILLE 1. Three Topics (1) Precision and Recall Curves. Receiver Operating Characteristic Curves (ROC). What to do if we do not fix the loss function? (2) The Curse of Dimensionality.

More information

Density Estimation (II)

Density Estimation (II) Density Estimation (II) Yesterday Overview & Issues Histogram Kernel estimators Ideogram Today Further development of optimization Estimating variance and bias Adaptive kernels Multivariate kernel estimation

More information

ECON 721: Lecture Notes on Nonparametric Density and Regression Estimation. Petra E. Todd

ECON 721: Lecture Notes on Nonparametric Density and Regression Estimation. Petra E. Todd ECON 721: Lecture Notes on Nonparametric Density and Regression Estimation Petra E. Todd Fall, 2014 2 Contents 1 Review of Stochastic Order Symbols 1 2 Nonparametric Density Estimation 3 2.1 Histogram

More information

Maximum Likelihood Estimation. only training data is available to design a classifier

Maximum Likelihood Estimation. only training data is available to design a classifier Introduction to Pattern Recognition [ Part 5 ] Mahdi Vasighi Introduction Bayesian Decision Theory shows that we could design an optimal classifier if we knew: P( i ) : priors p(x i ) : class-conditional

More information

Announcements. Proposals graded

Announcements. Proposals graded Announcements Proposals graded Kevin Jamieson 2018 1 Bayesian Methods Machine Learning CSE546 Kevin Jamieson University of Washington November 1, 2018 2018 Kevin Jamieson 2 MLE Recap - coin flips Data:

More information

A Novel Nonparametric Density Estimator

A Novel Nonparametric Density Estimator A Novel Nonparametric Density Estimator Z. I. Botev The University of Queensland Australia Abstract We present a novel nonparametric density estimator and a new data-driven bandwidth selection method with

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate

More information

Additive Isotonic Regression

Additive Isotonic Regression Additive Isotonic Regression Enno Mammen and Kyusang Yu 11. July 2006 INTRODUCTION: We have i.i.d. random vectors (Y 1, X 1 ),..., (Y n, X n ) with X i = (X1 i,..., X d i ) and we consider the additive

More information

GALAXY GROUPS IN THE THIRD DATA RELEASE OF THE SLOAN DIGITAL SKY SURVEY

GALAXY GROUPS IN THE THIRD DATA RELEASE OF THE SLOAN DIGITAL SKY SURVEY The Astrophysical Journal, 630:759 763, 2005 September 10 # 2005. The American Astronomical Society. All rights reserved. Printed in U.S.A. GALAXY GROUPS IN THE THIRD DATA RELEASE OF THE SLOAN DIGITAL

More information

Excerpts from previous presentations. Lauren Nicholson CWRU Departments of Astronomy and Physics

Excerpts from previous presentations. Lauren Nicholson CWRU Departments of Astronomy and Physics Excerpts from previous presentations Lauren Nicholson CWRU Departments of Astronomy and Physics Part 1: Review of Sloan Digital Sky Survey and the Galaxy Zoo Project Part 2: Putting it all together Part

More information

Inferring Galaxy Morphology Through Texture Analysis

Inferring Galaxy Morphology Through Texture Analysis Inferring Galaxy Morphology Through Texture Analysis 1 Galaxy Morphology Kinman Au, Christopher Genovese, Andrew Connolly Galaxies are not static objects; they evolve by interacting with the gas, dust

More information

Cosmology in the Very Local Universe - Why Flow Models Matter

Cosmology in the Very Local Universe - Why Flow Models Matter Cosmology in the Very Local Universe - Why Flow Models Matter Karen L. Masters Department of Astronomy, Cornell University, Ithaca, NY, 14853, USA While much of the focus of observational cosmology is

More information

Lectures in AstroStatistics: Topics in Machine Learning for Astronomers

Lectures in AstroStatistics: Topics in Machine Learning for Astronomers Lectures in AstroStatistics: Topics in Machine Learning for Astronomers Jessi Cisewski Yale University American Astronomical Society Meeting Wednesday, January 6, 2016 1 Statistical Learning - learning

More information

Mean-Shift Tracker Computer Vision (Kris Kitani) Carnegie Mellon University

Mean-Shift Tracker Computer Vision (Kris Kitani) Carnegie Mellon University Mean-Shift Tracker 16-385 Computer Vision (Kris Kitani) Carnegie Mellon University Mean Shift Algorithm A mode seeking algorithm Fukunaga & Hostetler (1975) Mean Shift Algorithm A mode seeking algorithm

More information

Data Analysis I. Dr Martin Hendry, Dept of Physics and Astronomy University of Glasgow, UK. 10 lectures, beginning October 2006

Data Analysis I. Dr Martin Hendry, Dept of Physics and Astronomy University of Glasgow, UK. 10 lectures, beginning October 2006 Astronomical p( y x, I) p( x, I) p ( x y, I) = p( y, I) Data Analysis I Dr Martin Hendry, Dept of Physics and Astronomy University of Glasgow, UK 10 lectures, beginning October 2006 4. Monte Carlo Methods

More information

A tailor made nonparametric density estimate

A tailor made nonparametric density estimate A tailor made nonparametric density estimate Daniel Carando 1, Ricardo Fraiman 2 and Pablo Groisman 1 1 Universidad de Buenos Aires 2 Universidad de San Andrés School and Workshop on Probability Theory

More information

More on Estimation. Maximum Likelihood Estimation.

More on Estimation. Maximum Likelihood Estimation. More on Estimation. In the previous chapter we looked at the properties of estimators and the criteria we could use to choose between types of estimators. Here we examine more closely some very popular

More information

Department of Statistics Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA citizenship: U.S.A.

Department of Statistics Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA citizenship: U.S.A. Chad M. Schafer Department of Statistics Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA 15213 cschafer@cmu.edu citizenship: U.S.A. Positions Held Associate Professor Department of Statistics,

More information

Preface. 1 Nonparametric Density Estimation and Testing. 1.1 Introduction. 1.2 Univariate Density Estimation

Preface. 1 Nonparametric Density Estimation and Testing. 1.1 Introduction. 1.2 Univariate Density Estimation Preface Nonparametric econometrics has become one of the most important sub-fields in modern econometrics. The primary goal of this lecture note is to introduce various nonparametric and semiparametric

More information

Approximate Bayesian Computation for the Stellar Initial Mass Function

Approximate Bayesian Computation for the Stellar Initial Mass Function Approximate Bayesian Computation for the Stellar Initial Mass Function Jessi Cisewski Department of Statistics Yale University SCMA6 Collaborators: Grant Weller (Savvysherpa), Chad Schafer (Carnegie Mellon),

More information

Data Modeling & Analysis Techniques. Probability & Statistics. Manfred Huber

Data Modeling & Analysis Techniques. Probability & Statistics. Manfred Huber Data Modeling & Analysis Techniques Probability & Statistics Manfred Huber 2017 1 Probability and Statistics Probability and statistics are often used interchangeably but are different, related fields

More information

Automatic Local Smoothing for Spectral Density. Abstract. This article uses local polynomial techniques to t Whittle's likelihood for spectral density

Automatic Local Smoothing for Spectral Density. Abstract. This article uses local polynomial techniques to t Whittle's likelihood for spectral density Automatic Local Smoothing for Spectral Density Estimation Jianqing Fan Department of Statistics University of North Carolina Chapel Hill, N.C. 27599-3260 Eva Kreutzberger Department of Mathematics University

More information

Marginal Specifications and a Gaussian Copula Estimation

Marginal Specifications and a Gaussian Copula Estimation Marginal Specifications and a Gaussian Copula Estimation Kazim Azam Abstract Multivariate analysis involving random variables of different type like count, continuous or mixture of both is frequently required

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning 3. Instance Based Learning Alex Smola Carnegie Mellon University http://alex.smola.org/teaching/cmu2013-10-701 10-701 Outline Parzen Windows Kernels, algorithm Model selection

More information

COMP 551 Applied Machine Learning Lecture 20: Gaussian processes

COMP 551 Applied Machine Learning Lecture 20: Gaussian processes COMP 55 Applied Machine Learning Lecture 2: Gaussian processes Instructor: Ryan Lowe (ryan.lowe@cs.mcgill.ca) Slides mostly by: (herke.vanhoof@mcgill.ca) Class web page: www.cs.mcgill.ca/~hvanho2/comp55

More information

Stat 5101 Lecture Notes

Stat 5101 Lecture Notes Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random

More information