Multinomial likelihood. Multinomial MLE. NIST data and genetic fingerprints. θ = (p 1,..., p m ) with j p j = 1 and p j 0. Point probabilities

Size: px

Start display at page:

Download "Multinomial likelihood. Multinomial MLE. NIST data and genetic fingerprints. θ = (p 1,..., p m ) with j p j = 1 and p j 0. Point probabilities"

Aron Richard
5 years ago
Views:

1 Multiomial distributio Let Y,..., Y be iid, uiformly sampled from a fiite populatio ad X i deotes a property of the idividual i. Label the properties,..., m. p j = PX i = j) = umber of idividuals with property j total umber of idividuals If X,..., X are iid from {,..., m} with PX i = j) = p j the with N j = X i = j) hx,..., X ) = N,..., N m ) {0,..., } m has the multiomial distributio. All outcomes x,..., x ) h {,..., m )}) have the same probability p pm m thus PN,..., N m ) =,..., m )) = h {,..., m )}) p }{{} pm m. The multiomial coefficiet Slide /22 Niels Richard Hase Statistics BI/E December 2, 9 Multiomial likelihood Sceario : X,..., X iid from {,..., m}, parameter θ = p,..., p m ) with j p j = ad p j 0. Poit probabilities PX i = x) = px) = p x Sample x = x,..., x ) {,..., m}, likelihood fuctio with j = i x i = j). L x θ) = px ) px 2 ) px ) = p pm m Sceario 2: N = N,..., N m ) sample from the multiomial distributio o E = {0,..., }, parameter θ = p,..., p m ) with j p j = ad p j 0. Sample x =,..., m ) E, likelihood fuctio ) L x θ) = p... pm m m I both cases the mius-log-likelihood equals m j logp j ) j= up to a parameter idepedet term... Slide 2/22 Niels Richard Hase Statistics BI/E December 2, 9 Multiomial MLE It is possible to fid the maximum of the multiomial likelihood ad ot surprisigly it is ˆp j = j The relative frequecy of the idividuals i the sample havig property j is the atural estimate ad i cocordace with the frequecy iterpretatio of probabilities. It does ot matter for the MLE if we have sceario or 2 above. From the poit of view of the likelihood ad parameter estimatio the table of summarized couts N j = X i = j) cotais all iformatio about the ukow parameter. NIST data ad geetic figerprits The NIST dataset cotais data from a sample of the America populatio. There are three subpopulatios, Caucasias, Hispaics ad Africa-Americas. For each idividual a umber of differet properties are recorded at the same time. A short tadem repeat STR) is a geetic sigal that shows some variatio across the populatio, which ca make it useful for discrimiative purposes. There are 5 differet STRs recorded i the dataset, for each there is a two-dimesioal cout, 2 ) with 2 givig the umber of repeat couts o the two chromosomes. For a sigle STR we have a multiomial model, the properties are the differet ordered couts, 2 ) that we ca observe. Slide 3/22 Niels Richard Hase Statistics BI/E December 2, 9 Slide 4/22 Niels Richard Hase Statistics BI/E December 2, 9

2 Hardy-Weiberg equilibrium All diploid orgaisms like humas carry two copies of the chromosomes. For a gee occurrig i two combiatios as allele A or a there are three possible geotypes: AA, Aa, aa. We sample a radom idividual from a populatio ad observe Y the geotype takig values i the sample space E = {AA, Aa, aa}. With X f ad X m deote uobservable father ad mother alleles for the radom idividual the Y is a trasformatio of these. Uder a radom matig assumptio idepedece of X m ad X f ad equal distributio assumptio i the male ad female populatios: Exercise - NIST data ad Hardy-Weiberg Dowload the NIST data. Compute the MLE for the STR D6S539 for the etire dataset. Compute the MLE for the Caucasia populatio oly. Fid the likelihood fuctio for the D6S539 couts uder the Hardy-Weiberg equilibrium assumptio ad compute the MLE uder this model. PY = AA) = p 2, PY = Aa) = 2p p), PY = aa) = p) 2 with p = PX m = A) = PX f = A). Slide 5/22 Niels Richard Hase Statistics BI/E December 2, 9 Slide 6/22 Niels Richard Hase Statistics BI/E December 2, 9 Expoetial distributio locatio-scale model Itroducig a locatio parameter i the expoetial distributio ad makig iid observatios the likelihood becomes { λ L x µ, λ) = e λ P x i µ) µ < mi{x,..., x } 0 otherwise Fixig µ [0, mi{x,..., x }) the uique) maximizer i λ is ˆλµ) = x ) The profile likelihood fuctio is L x µ, ˆλµ)) = ˆλµ) e = x e )) Scale-locatio i the Gumbel distributio Sample space E = R, parameter space Θ = R 0, ), ad parameters θ = µ, ). The probability measure P θ has desity f µ, x,..., x ) = exp x exp x )). The mius-log-likelihood fuctio for observig x = x,..., x ) is l x µ, ) = log + x + exp x ). ) This is the scale-locatio model for the Gumbel distributio. which is mootoely icreasig i µ. There is o MLE. Slide 7/22 Niels Richard Hase Statistics BI/E December 2, 9 Slide 8/22 Niels Richard Hase Statistics BI/E December 2, 9

3 " Reparameterized mius-log-likelihood l x µ, ) = log + x + Useful trick; reparameterize µ η, ρ) =, ) R 0, ). l x η, ρ) = = ρ ρx i η) + x i + expη) = ρx + expη) exp x ). exp ρx i + η) log ρ exp ρx i ) η log ρ exp ρx i ) η log ρ Miimizatio over η The likelihood equatio has solutio dη η, ρ) = expη) exp ρx i ) = 0 ˆηρ) = log ) exp ρx i ). Secod differetiatio shows that this is a uique, global miimum for each fixed ρ > 0. x = x i Slide 9/22 Niels Richard Hase Statistics BI/E December 2, 9 Slide 0/22 Niels Richard Hase Statistics BI/E December 2, 9 The mius-log-likelihood The profile mius-log-likelihood fuctio l x µ, ) = log + ˆηρ) = log x + ) exp ρx i ). exp x ) Figure: The mius-log-likelihood fuctio left) ad a cotour plot right) for the η, ρ) reparameterizatio of the scale-locatio model based o = 00 simulatios of iid Gumbel distributed variables. The MLE is ˆη = ad ˆρ = The profile mius-log-likelihood fuctio of ρ is give by evaluatig the mius-log-likelihood fuctio alog the curved lie, as show o the cotour plot, give by the equatio ˆηρ) = η. Slide /22 Niels Richard Hase Statistics BI/E December 2, l x ˆηρ), ρ) = ρx + + log ) exp ρx i ) log ρ, The profile) likelihood equatio becomes x i exp ρx i ) exp ρx + i) ρ = x Slide 2/22 Niels Richard Hase Statistics BI/E December 2, 9

4 Profile mius-log-likelihood fuctio Coclusios profile mius log likelihood fuctio The secod derivative of l x ˆηρ), ρ) is strictly positive, hece there ca oly be oe solutio to the profile) likelihood equatio i ρ but if there are at least two differet observatios there will be a solutio. If there are at least two differet observatios there will be a uique maximum of the likelihood fuctio, which ca be foud by solvig the uivariate o-liear) equatio i ρ umerically Figure: The profile mius-log-likelihood fuctio for ρ with 00 simulated Gumbel variables. Slide 3/22 Niels Richard Hase Statistics BI/E December 2, 9 Slide 4/22 Niels Richard Hase Statistics BI/E December 2, 9 The Newto-Raphso algorithm Likelihood equatio: θ) = 0. dθ First order Taylor expasio dθ θ) dθ θ 0) + d2 l x d 2 θ θ 0)θ θ 0 ). Solve for θ 0 Θ the liear equatio dθ θ 0) + d2 l x d 2 θ θ 0)θ θ 0 ) = 0 With iitial guess θ 0 the iterative solutios d 2 ) l x θ = θ d 2 θ θ dlx ) dθ θ ). gives the Newto-Raphso algorithm. Slide 5/22 Niels Richard Hase Statistics BI/E December 2, 9 Fly death Cocetratio logcocetratio) Deaths Survivors The umber of survivig flies depedig o the cocetratio of the isecticide dimethoat. Slide 6/22 Niels Richard Hase Statistics BI/E December 2, 9

5 Logistic regressio Fly-death is a 0--variable a Beroulli variable), whose distributio depeds upo the cocetratio of the isecticide. We itroduce the poit probability that a fly dies X i = ) as py) = expα + βy) + expα + βy), Mius-log-likelihood fuctio where py) 0, ) ad α, β > 0. model. The likelihood fuctio is This is the logistic regressio Mius log likelihood L x α, β) = py i ) x i py i )) x i = ad the mius-log-likelihood fuctio becomes l x α, β) = expαx i + βy i x i ) + expα + βy i ) log + expα + βy i )) αs βss, 00 beta alpha Slide 7/22 Niels Richard Hase Statistics BI/E December 2, 9 Slide 8/22 Niels Richard Hase Statistics BI/E December 2, 9 Mius-log-likelihood ad iteratios Mius-log-likelihood ad iteratios " " Slide 9/22 Niels Richard Hase Statistics BI/E December 2, 9 Slide 20/22 Niels Richard Hase Statistics BI/E December 2, 9

6 Mius-log-likelihood ad iteratios Logistic regressio curve - fly-death ".5 5*6263)3078*98:/20; %&% %&# %& %&' %& $&% 00 " # $ % )*+,-*.-/.0203*.4 Slide 2/22 Niels Richard Hase Statistics BI/E December 2, 9 Slide 22/22 Niels Richard Hase Statistics BI/E December 2, 9

Chapter 6 Principles of Data Reduction

Chapter 6 Principles of Data Reduction Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a