Part I: Bacgroud o the Bomal Dstrbuto A radom varable s sad to have a Beroull dstrbuto f t taes o the value wth probablt "p" ad the value wth probablt " - p". The umber of "successes" "" depedet Beroull trals s sad to have a bomal dstrbuto. The probablt mass fucto for the bomal dstrbuto s: probablt(observg successes, p) = choose(, ) * p^ * (-p)^(-) Let's cosder a cocrete example. Suppose we wat to ow the probablt of observg "" heads "" tosses of a co, where the probablt of ladg o heads s "p" = /2. The co tosses are depedet, because the outcome for the prevous co toss does ot affect the outcome for the ext co toss. The co tosses are detcall dstrbuted, because the probablt of ladg o heads s the same for all co tosses. There are eght possble outcomes: tals, tals, tals: probablt of ths outcome = ( - p) * ( - p) * ( - p) = /8 tals, tals, heads: probablt of ths outcome = ( - p) * ( - p) * p = /8 tals, heads, tals: probablt of ths outcome = ( - p) * p * ( - p) = /8 tals, heads, heads: probablt of ths outcome = ( - p) * p * p = /8 heads, tals, tals: probablt of ths outcome = p * ( - p) * ( - p) = /8 heads, tals, heads: probablt of ths outcome = p * ( - p) * p = /8 heads, heads, tals: probablt of ths outcome = p * p * ( - p) = /8 heads, heads, heads: probablt of ths outcome = p * p * p = /8
Part II: Logstc Regresso Usg Smulated Data As ou ma recall, the bomal dstrbuto s our source of ucertat for logstc regresso. Ths s aalogous to usg the Gaussa (ormal) dstrbuto as the source of ucertat for lear regresso.
Part III: Estmatg a Bomal Proporto Suppose we have a set of Beroull outcomes, ad we wat to estmate the bomal proporto from data (.e. we wat to estmate "p"). We ca geerate a "maxmum lelhood estmate" for the bomal proporto b settg the dervatve of the log lelhood to zero ad solvg for "p". probablt, so tag the log of both sdes gves us log, log probablt log log log ad tag the dervatve wth respect to theta gves us log probablt, log log log log log log ad settg the dervatve equal to zero ad solvg for theta gves us
Ths estmate s sometmes called a "frequetst" estmate of the bomal proporto. The problem wth ths estmate occurs whe the actual bomal proporto s ether ver small or ver large, or whe s small. Suppose the actual bomal proporto s p =. ad we ol have observatos. The expected umber of "successes" ( * p = *. =.) s less tha oe. Ths meas our frequetst estmate s lel to be, eve though the actual bomal proporto s.. Ths ca cause trouble a doma where observed probabltes ca be ver small or ver large; e.g. whe estmatg clc-through rates for ole advertsg or purchase probabltes for tems the "tal" of a ecommerce merchat's vetor. We would le to avod degeerate estmates that express the certat of ether zero or oe. Baesa estmato ca be used to adjust the frequetst estmate. Recall that posteror = pror * lelhood / evdece. pror lelhood p, /,,, dbeta probablt evdece p,,, posteror p,,, / d / / d /,, dbeta
The expected value (mea) of the posteror dstrbuto s smpl ( + alpha ) / ( + alpha + (-) + alpha ). We are essetall adjustg the frequetst estmate b addg alpha ad alpha pseudo couts to the ad - observed couts. Usg the posteror mea for our estmate of the bomal proporto s the same as usg a weghted mea of ( / ) ad (alpha / (alpha + alpha )), so we ll wat to eep the pseudo couts relatvel small to allow the observed couts to overtae the pseudo couts relatvel qucl ˆ We re usg a Baesa pror to avod the certat of usg ether zero or oe for our estmate of the bomal proporto. I practce, fols ofte use ether alpha = alpha = (a Laplace pror) or alpha = alpha =.5 (a Jeffres pror). We ca obta a 95% cofdece terval for the bomal proporto wth the followg R commad qbeta(c(.25,.975), + alpha., - + alpha.) Note: the R bom.test() commad uses the followg slghtl more coservatve (larger) terval c(qbeta(.25,, - + ), qbeta(.975, +, - ))