Lesson 6 Maximum Likelihood: Part III. (plus a quick bestiary of models)

Transcription

1 Lesson 6 Maximum Likelihood: Part III (plus a quick bestiary of models)

2 Homework You want to know the density of fish in a set of experimental ponds You observe the following counts in ten ponds: 5,6,7,3,6,5,8,4,4,3 What is your process model? What is your data model? Solve for the analytical MLE What is the estimate for this population?

3 Process model: f(x) = l (density) Data model: x ~ Pois(l)

4 How the likelihood is constructed L = Pr(data model parameters) How is the data modeled? What type of data? What process generated this data? What distributions are an appropriate description of the data? How is the process modeled? Each analysis should be approached individually Problem solving, creativity

5 How is the data modeled? What type of data is it: Continuous Integer / Count Boolean (0/1) Factor / categorical Are there range restrictions on the data? Are negative values allowed? Is there an upper bound? Are the observed data near the bounds?

6 How is the data modeled? What are the dominant sources of variability in the data? Observation/measurement error Process variability Space ime Individual/Site/Species Random Missing data

7 How is the data modeled? Are there multiple processes involved? Zero-inflated data Pr(abundance present)pr(present) Are there multiple types or sources of data? ree growth: tree rings + DBH ree fecundity: cone counts + seed trap ree crown: remote sensing + model + crown class Is the process observed directly or inferred?

8 How is the process modeled? Constant mean Multiple means by factor (ANOVA) As a function of covariates Linear models Generalized linear models Nonlinear models Hierarchical models Mechanistic models Note: Will shy away from data mining models except for EDA: regression trees, splines, neural networks, etc.

9 A quick beastiary of functions Polynomials Piecewise polynomials Rational (ratio based) Exponential based Power-based Sometimes chosen for mechanistic reasons, sometimes because they fit right Supplemental reading: Bolker Ch 3

10 Polynomial Linear with respect to the model parameters aylor series: Can approximate any smooth continuous function

11 Piecewise AKA change point analysis

12 Rational

13 Exponential

14 Power

15

16 Linear Regression y i =a 0 a 1 x i i i ~N 0, 2 Sections & 5.4.2

17 Linear Regression Step 1: Likelihood n L= i=1 = y i =a 0 a 1 x i i i ~N 0, 2 N y i a 0 a 1 x i, i 1 2 n exp[ y i a 0 a 1 x i 2] Sections & 5.4.2

18 Intercept 1 L= 2 exp[ n ln L= n ln n ln y i a 0 a 1 x i 2] y i a 0 a 1 x i 2

19 Intercept 1 L= 2 exp[ 2] n 1 y 2 2 i a 0 a 1 x i ln L= n ln n ln 2 1 y 2 2 i a 0 a 1 x i 2 ln L = 1 y a 0 2 i a 0 a 1 x i

20 Intercept 1 L= 2 exp[ 2] n 1 y 2 2 i a 0 a 1 x i ln L= n ln n ln 2 1 y 2 2 i a 0 a 1 x i 2 ln L = 1 y a 0 2 i a 0 a 1 x i 0 = y i n a 0 a 1 x i

21 Intercept 1 L= 2 exp[ 2] n 1 y 2 2 i a 0 a 1 x i ln L= n ln n ln 2 1 y 2 2 i a 0 a 1 x i 2 ln L = 1 y a 0 2 i a 0 a 1 x i 0 = y i n a 0 0 = y a 0 a 1 x a 1 x i

22 Intercept 1 L= 2 exp[ 2] n 1 y 2 2 i a 0 a 1 x i ln L= n ln n ln 2 1 y 2 2 i a 0 a 1 x i 2 ln L = 1 y a 0 2 i a 0 a 1 x i 0 = y i n a 0 0 = y a 0 a 1 x a 0 = y a 1 x a 1 x i

23 Slope ln L= n ln n ln y i a 0 a 1 x i 2

24 Slope ln L= n ln n ln ln L = 1 x a 1 2 i y i a 0 a 1 x i y i a 0 a 1 x i 2

25 Slope ln L= n ln n ln ln L a 1 = 0 = 1 2 x i y i a 0 a 1 x i x i y i a 0 x i a 1 x i 2 y i a 0 a 1 x i 2

26 Slope ln L= n ln n ln ln L a 1 = 1 2 x i y i a 0 a 1 x i 0 = x i y i 0 = xy a 0 x a 1 x 2 a 0 x i a 1 x i 2 y i a 0 a 1 x i 2

27 Slope ln L= n ln n ln ln L a 1 = 1 2 x i y i a 0 a 1 x i 0 = x i y i 0 = xy a 0 x a 1 x 2 a 1 = a 0 x i xy a 0 x x 2 a 1 x i 2 y i a 0 a 1 x i 2

28 Combining slope and intercept a 0 = y a 1 x a 1 = xy a 0 x x 2

29 Combining slope and intercept a 0 = y a 1 x a 1 = xy a 0 x x 2 a 1 = xy x y a 1 x 2 x 2

30 Combining slope and intercept a 0 = y a 1 x a 1 = xy a 0 x x 2 a 1 = xy x y a 1 x 2 x 2 a 1 = xy x y cov [x, y] = x 2 2 x var [x]

31 Combining slope and intercept a 0 = y a 1 x a 1 = xy a 0 x x 2 a 1 = xy x y a 1 x 2 x 2 a 1 = xy x y cov [x, y] = x 2 2 x var [x] a 0 = x2 y x xy var [x]

32 Variance ln L= n ln n ln y i a 0 a 1 x i 2

33 Variance ln L= n ln n ln 2 1 y 2 2 i a 0 a 1 x i 2 ln L = n 1 n y 3 i a 0 a 1 x i 2 = 0 i=1

34 Variance ln L= n ln n ln 2 1 y 2 2 i a 0 a 1 x i 2 ln L = n 1 n y 3 i a 0 a 1 x i 2 = 0 i=1 n = 1 n 3 i=1 y i a 0 a 1 x i 2

35 Variance ln L= n ln n ln 2 1 y 2 2 i a 0 a 1 x i 2 ln L = n 1 n y 3 i a 0 a 1 x i 2 = 0 i=1 n = 1 n 3 i=1 y i a 0 a 1 x i 2 2 ML n = 1 n i=1 y i a 0 a 1 x i 2

36 Matrix notation y i = 1 2 x i x i =[ x i1 x i2] [ 1 2] = 1 x i1 2 x i2 Where x i1 = 1 X=[1 x 1 y=x 1 x 2 1 x n] Design Matrix

37 Design Matrices Multiple linear regression X=[1 x x 12 1k 1 x x 22 2k 1 x n2 x nk] a 1 = xy x y x 2 x 2 a 0 = x2 y x xy var [x] = X X 1 X y

38 ANOVA Design Matrices One Way Anova 3 levels, n=6 1] [ b 1 b 2 b 3 b 1 b 1 +b 2 b 1 +b 3

39 ANOVA Design Matrices One Way Anova 3 levels, n=6 1] [ wo Way Anova 3 levels x 2 levels 2 reps each, n=12 0] [ F C L1+F L1 L2+F L2 C L1 L2 F L1+F L2+F C L 1 L 2 F

40 ANOVA Design Matrices One Way Anova 3 levels, n=6 1] [ wo Way Anova 3 levels x 2 levels 2 reps each, n=12 0] [ [1 ANCOVA 2 levels, 1 covariate n=6 0 x x x x x x 63]