STA 114: Saisics Noes 2. Saisical Models and he Likelihood Funcion Describing Daa & Saisical Models A physicis has a heory ha makes a precise predicion of wha s o be observed in daa. If he daa doesn mach he predicion, hen he heory is falsified. A saisician only has an imprecise descripion. This could be eiher because he heory is imprecise, or because random errors are inroduced in collecing he daa, or a combinaion of he wo. Therefore a saisician s daa, from he perspecive of her heory + daa collecion mehod, is an uncerain quaniy X. Any uncerain quaniy can be bes described by a se of values S he quaniy may assume, wih a pdf/pmf f() on S. The pdf/pmf is o be inerpreed as follows: f( 1 )/f( 2 ) = r means ha X = 1 is r-imes as plausible as X = 2. If he daa can be described by a single pmf/pdf hen here is no need of saisical analysis. Saisics is needed when a muliude of compeing heories lead o a muliude of pmfs/pdfs. When all hese pmfs/pdfs are colleced ogeher, we have a saisical model for our analysis. If θ denoes he quaniy by which he consiuen pmfs/pdfs of he model differ from each oher, hen we can wrie each pmf/pdf as f( θ). The quaniy θ is a parameer of his model. The se Θ of all possible values of θ is called he parameer space of he model. Eample (Opinion Poll). Take for eample a sudy where one wans o know wha percenage of sudens in a cerain universiy are in favor of a recen governmen policy. For a large universiy, soliciing every suden s opinion is impossible. The researcher may wan o draw a random lis of n = 5 sudens and quiz hem on heir opinion regarding he policy. A random lis gives he bes chance of guarding agains sysemaic biases in obaining a represenaive sample of sudens. The daa here is he number X of sudens in he sample who are in favor. If he researcher hinks ha a fracion p of he sudens, among a oal of N universiy sudens are in favor of he policy, hen X can be described as hyper-geomeric pmf f( p) given by { ( m )( N m n ) for =, 1, 2,, min(n, m) f( p) = ( N n) oherwise where m = Np is he oal number of sudens in he universiy who are in favor of he policy. The fracion p represens he researcher s heory abou he populariy of he policy among college sudens. If she considers all possibiliies p 1, hen here saisical model for X is {f( p) : p [, 1]} wih f( p) given as above. 1
HypGeo Binom f( p)..2.4 f( p)..2.4 4 4 Figure 1: X = number of sudens favoring he policy in a sample of 5 sudens. Descripion of X under hypergeomeric (lef) and binomial disribuions (righ) for hree possible values of p =.25,.5,.75. When N is very large compared o n, we can also represen X by he binomial pmf { ( n ) p (1 p) n for =, 1, 2,, n f( p) = oherwise Now he researcher s model is {f( p) : p [, 1]} wih f( p) given by he binomial pmf above. Figure 1 below shows wha he researcher epecs o see as daa X under he hypergeomeric or he binomial disribuion for hree possible values of p, namely, p = 1/4 (solid line), p = 1/2 (broken line) and p = 3/4 (doed line). Eample (Trend of TC couns). A climae researcher wans o sudy wheher hurricane aciviy is inensifying wih ime. One way o do i is o sudy he annual couns of ropical cyclones (TC) in an ocean basin, say he norh Alanic basin, for he pas 1 years. The daa is hen of he form X = (X 1, X 2,, X 1 ), wih X giving he TC coun in year. To describe his daa, we can firs focus on describing one X. Since X is a coun, we can describe i by a Poisson pmf: { e µ µ f ( µ ) =! for =, 1, 2, oherwise where µ represens he epeced coun for year. Now o describe, X = (X 1, X 2,, X 1 ) we can rea he componen X s as independen and wrie f( {µ }) = f 1 ( 1 µ 1 ) f 2 ( 2 µ 2 ) f 1 ( 1 µ 1 ) which gives he join pmf of X a = ( 1, 2,, 1 ). Alhough he above gives a descripion of X, i is no clear how o sudy he climae researcher s quesion wihin his framework. To achieve his, we now need o say somehing 2
4 6 8 1 5 1 3 4 5 1 3 4 5 4 3 1 4 6 8 1 4 6 8 1 Figure 2: X = annual TC couns for 1 consecuive years. Descripion of X under Poisson disribuions wih mean µ in year. Three possible linear specificaions µ = µ + β( 1) are considered. abou how he differen µ compare o each oher, and in paricular, how hey evolve over ime. One possible descripion is he following: µ = α + β( 1), = 1, 2,, 1 which says ha he epeced annual couns are increasing linearly in ime, wih slope β. The research quesion of wheher TC aciviy is increasing can now be represened by various values of (α, β). In paricular, a posiive sign of β means ha TC couns have an upward rend, wih larger β indicaing faser growh. On he oher hand, a zero or a negaive value of β indicaes no or downward rend. Therefore a saisical model for X is given by {f ( µ, β) : α (a, b), β (c, d)} for well chosen limis a, b, c, d, where f ( α, β) = f1 (1 α) f2 (2 α + β) f1 (1 α + 99β). Figure 2 shows he descripion of X under hree choices of (α, β): (,.2), (1,.25) and (, ). Noe ha unlike he previous eample, he he choice of model for his eample was a lo less obvious. Indeed, one could use many disribuions, insead of a Poisson pmf, o describe each X. Furhermore, he evoluion of µ over ime, could also be described in many differen ways. Wha we have buil here is a descripion of he daa, wheher here is a beer descripion can always be debaed. The Likelihood Funcion Suppose a saisical model {f ( θ) : θ Θ} has been consruced for daa X, wih each θ represening a differen heory. When we observed daa X =, we can compare wo parameer values (i.e., wo heories) θ = θ1 and θ = θ2 by looking a he raio f ( θ1 )/f ( θ2 ). If his raio equals 2, hen he daa X = is wice as likely o be observed under θ = θ1 han i is under θ = θ2. Such comparisons can be done based on he likelihood funcion L (θ) := f ( θ), θ Θ. 3
Likelihood..2 Log-likelihood -1-6 -..4.8 p..4.8 p Figure 3: Likelihood and log-likelihood funcions in he opinion poll eample. The observed daa is X =. Noe ha L (θ) is a funcion over he variable θ aking values in he se Θ. For all echnical purposes, one can work wih L (θ) in he log-scale. Tha is, define he log-likelihood funcion l (θ) = log L (θ) = log f( θ). Log-scale comparisons beween heories are hen done by differences l (θ 1 ) l (θ 2 ). Eample (Opinion Poll, Cond). For he opinion poll eample wih he saisical model {Binomial(n, p) : p [.1]}, he likelihood funcion in he parameer p is given by ( ) n L (p) = p (1 p) n, p [, 1] and he log-likelihood funcion is l (p) = log L (p) = log ( ) n + log p + (n ) log(1 p), p [, 1] Noe ha he firs erm on he righ hand side does no involve he funcion argumen p. So we can wrie l (p) = cons + log p + (n ) log(1 p), no caring abou he eac value of his addiive consan. Indeed, he consan disappears when we look a differences l (p 1 ) l (p 2 ). For daa X = he heories p =.25, p =.5 and p =.75 receive likelihood scores 6.45 1 14, 1.54 1 6 and 1.25 1 61. Figure 3 shows he likelihood funcion L (p) and he log-likelihood funcion l (p) over he grid p {.,.1,, 1.}. These funcions indicae ha heories wih p close o.4 fare well in eplaining he daa X =. The heory p =.4 eplains he daa nearly 1 8 imes beer han he heory p =.8. 4
1 3 4 5 6 1 -.5.5.15.25 5 1 15 Figure 4: TC couns beween 198 and 7 and he corresponding log-likelihood funcion shown as a conour. Eample (TC couns, Cond.). For he saisical model we discussed before, he loglikelihood funcion is given by: 1 1 l (α, β) = cons (α + β( 1)) + log(α + β( 1)). =1 Figure 4 shows he observed annual TC couns beween 198 and 7 (on he lef superimposed on f( 7,.5)). A conour plo of he he log-likelihood funcion over (α, β) is shown on he righ. Posiive slope values (β > ) fare beer in eplaining he daa han negaive slopes. A Word of Cauion The likelihood funcion gives a numerical comparison of he posulaed heories once daa X = is observed. Bu be clear on wha L (θ 1 )/L (θ 2 ) = 2 means. I does NOT mean ha given he observed daa, heory θ 1 is wice more likely han heory θ 1. We don ye have a plaform for discussing likeliness or relaive plausibiliy of heories. Formalizing his concep is he focus of saisical inference. =1 5