The EbayesThresh Package

Transcription

1 The EbyesThresh Pckge Mrch 26, 2005 Title Empiricl Byes thresholding nd relted methods Version Dte Author Bernrd W. Silvermn Mintiner Bernrd W. Silvermn This pckge crries out Empiricl Byes thresholding using the methods developed by I. M. Johnstone nd B. W. Silvermn. The bsic problem is to estimte men vector given vector of observtions of the men vector plus white noise, tking dvntge of possible sprsity in the men vector. Within Byesin formultion, the elements of the men vector re modelled s hving, independently, distribution tht is miture of n tom of probbility t zero nd suitble hevy-tiled distribution. The miing prmeter cn be estimted by mrginl mimum likelihood pproch. This leds to n dptive thresholding pproch on the originl dt. Etensions of the bsic method, in prticulr to wvelet thresholding, re lso implemented within the pckge. License GPL version 2 or newer URL R topics documented: bet.cuchy bet.lplce ebyesthresh ebyesthresh.wvelet isotone postmen postmed tfromw tfrom threshld vecbinsolv wndfrom wfromt wfrom wmonfrom zetfrom

2 2 bet.lplce Inde 20 bet.cuchy Function bet for the qusi-cuchy Given vlue or vector of vlues, find the vlue(s) of the function β() = g()/φ() 1, where g is the convolution of the qusi-cuchy with the norml density φ(). bet.cuchy() rel vlue or vector A vector the sme length s, contining the vlue(s) β(). bet.lplce Emples bet.cuchy(c(-2,1,0,-4,8,50)) bet.lplce Function bet for the Lplce Given vlue or vector of vlues, find the vlue(s) of the function β() = g()/φ() 1, where g is the convolution of the Lplce density with scle prmeter with with the norml density φ(). bet.lplce(, = 0.5)

3 ebyesthresh 3 the vlue or vector of dt vlues the scle prmeter of the Lplce distribution Note A vector the sme length s is returned, contining the vlue(s) β(). The Lplce density is given by γ(u) = 1 2 e u nd is lso known s the double eponentil density. bet.cuchy Emples bet.lplce(c(-2,1,0,-4,8,50)) bet.lplce(c(-2,1,0,-4,8,50), =1) ebyesthresh Empiricl Byes thresholding on sequence Given sequence of dt, performs Empiricl Byes thresholding, s discussed in Johnstone nd Silvermn (2004). ebyesthresh(, = "lplce", = 0.5, byesfc = FALSE, sdev = NA, verbose = FALSE, threshrule = "medin") vector of dt vlues specifiction of to be used conditionl on the men being nonzero; cn be cuchy or lplce scle fctor if Lplce is used. Ignored if Cuchy is used. If, on entry, =NA nd ="lplce", then the scle prmeter will lso be estimted by mrginl mimum likelihood. If is not specified then the defult vlue 0.5 will be used.

4 4 ebyesthresh byesfc sdev verbose threshrule if byesfc=true, then whenever threshold is eplicitly clculted, the Byes fctor threshold will be used the smpling stndrd devition of the dt. If, on entry, sdev=na, then the stndrd devition will be estimted using the medin bsolute devition from zero, s md(, center=0). controls the level of output. See below. specifies the thresholding rule to be pplied to the dt. Possible vlues re medin (use the posterior medin); men (use the posterior men); hrd (crry out hrd thresholding); soft (crry out soft thresholding); none (find vrious prmeters, but do not crry out ny thresholding). Detils It is ssumed tht the dt vector ( 1,..., n ) is such tht ech i is drwn independently from norml distribution with men θ i nd vrince σ 2. The distribution of ech θ i is miture with probbility 1 w of zero nd probbility w of given symmetric hevy-tiled distribution. The miing weight w is estimted by mrginl mimum likelihood. Given the miing weight, nd possibly scle fctor in the symmetric distribution, re estimted by mrginl mimum likelihood. The resulting vlues re used s the hyperprmeters in the. The prmeters cn be estimted s the posterior medin or the posterior men given the dt, or by hrd or soft thresholding using the posterior medin threshold. If hrd or soft thresholding is chosen, then there is the dditionl choice of using the Byes fctor threshold, which is the vlue such tht the posterior probbility of zero is ectly hlf if the dt vlue is equl to the threshold. If verbose=false, vector giving the vlues of the estimtes of the underlying men vector. If verbose=true, list with the following elements: muht the estimted men vector (omitted if threshrule="none") the dt vector s supplied threshold.sdevscle the threshold s multiple of the stndrd devition sdev threshold.origscle the threshold mesured on the originl scle of the dt w byesfc sdev threshrule the tht ws used the miing weight s estimted by mrginl mimum likelihood (only present if Lplce used) the scle fctor s supplied or estimted the vlue of the prmeter byesfc, determining whether Byes fctor or posterior medin thresholds re used the stndrd devition of the dt s supplied or estimted the thresholding rule used, s specified bove

5 ebyesthresh.wvelet 5 Johnstone, I. M. nd Silvermn, B. W. (2004) Needles nd strw in hystcks: Empiricl Byes estimtes of possibly sprse sequences. Annls of Sttistics, 32, Johnstone, I. M. nd Silvermn, B. W. (2004) EbyesThresh: R softwre for Empiricl Byes thresholding. Journl of Sttisticl Softwre. To pper. Johnstone, I. M. (2004) Function Estimtion nd Clssicl Norml Theory The Threshold Selection Problem. The Wld Lectures I nd II, Avilble from stnford.edu/~imj/. Johnstone, I. M. nd Silvermn, B. W. (2005) Empiricl Byes selection of wvelet thresholds. Annls of Sttistics, 33, to pper. The ppers by Johnstone nd Silvermn re vilble from com. See lso for further references, including the drft of monogrph by I. M. Johnstone. tfrom, threshld Emples ebyesthresh(=rnorm(100, c( rep(0,90), rep(5,10))), ="cuchy", sdev=na) ebyesthresh.wvelet Empiricl Byes thresholding on the levels of wvelet trnsform Apply n Empiricl Byes thresholding pproch level by level to the detil coefficients in wvelet trnsform. ebyesthresh.wvelet(tr, vscle = "independent", smooth.levels = Inf, = "lplce", = 0.5, byesfc = FALSE, threshrule = "medin") tr vscle The wvelet trnsform of vector of dt. The trnsform is obtined using one of the wvelet trnsform routines in R or in S+WAVELETS. Any choice of wvelet, boundry condition, etc provided by these routines cn be used. Controls the scle used t different levels of the trnsform. If vscle is sclr quntity, then it will be ssumed tht the wvelet coefficients t every level hve this stndrd devition. If vscle = "independent", the stndrd devition will be estimted from the highest level of the wvelet trnsform nd will then be used for ll levels processed. If vscle="level", then the stndrd devition will be estimted seprtely for ech level processed, llowing stndrd devition tht is level-dependent.

6 6 ebyesthresh.wvelet smooth.levels the number of levels to be processed, if less thn the number of levels of detil clculted by the wvelet trnsform. Detils byesfc threshrule specifiction of to be used for the coefficients t ech level, conditionl on their men being nonzero; cn be cuchy or lplce scle fctor if Lplce is used. Ignored if Cuchy is used. If, on entry, =NA nd ="lplce", then the scle prmeter will lso be estimted t ech level by mrginl mimum likelihood. If is not specified then the defult vlue 0.5 will be used. if byesfc=true, then whenever threshold is eplicitly clculted, the Byes fctor threshold will be used specifies the thresholding rule to be pplied to the coefficients. Possible vlues re medin (use the posterior medin); men (use the posterior men); hrd (crry out hrd thresholding); soft (crry out soft thresholding); The routine ebyesthresh.wvelet cn process wvelet trnsform obtined using the routine wd in the WveThresh R pckge, the routines dwt or modwt in the wveslim R pckge, or one of the routines (either dwt or nd.dwt) in S+WAVELETS. Note tht the wvelet trnsform must be clculted before the routine ebyesthresh.wvelet is clled; the choice of wvelet, boundry conditions, decimted vs nondecimted wvelet, nd so on, re mde when the wvelet trnsform is clculted. Aprt from some housekeeping to estimte the stndrd devition if necessry, nd to determine the number of levels to be processed, the min prt of the routine is cll, for ech level, to the smoothing routine ebyesthresh. The bsic notion of processing ech level of detil coefficients is esily trnsferred to trnsforms constructed using other wvelet softwre. Similrly, it is strightforwrd to modify the routine to give other detils of the wvelet trnsform, if necessry using the option verbose = TRUE in the clls to ebyesthresh. The min routine ebyesthresh.wvelet clls the relevnt one of the routines ebyesthresh.wvelet.wd (for trnsform obtined from WveThresh), ebyesthresh.wvelet.dwt (for trnsforms obtined from either dwt or modwt in wveslim) or ebyesthresh.wvelet.splus (for trnsforms obtined from S+WAVELETS. The wvelet trnsform (in the sme formt s tht supplied to the routine) of the vlues of the estimted regression function underlying the originl dt. Johnstone, I. M. nd Silvermn, B. W. (2005) Empiricl Byes selection of wvelet thresholds. Annls of Sttistics, 33, to pper. See lso the other references given for ebyesthresh nd t com ebyesthresh

7 isotone 7 isotone Weighted lest squres monotone regression Given vector of dt nd vector of weights, find the monotone sequence closest to the dt in the sense of weighted lest squres with the given weights. isotone(, wt = rep(1, length()), incresing = FALSE) wt incresing vector of dt vector the sme length s, giving the weights to be used in the weighted lest squres lgorithm logicl vrible indicting whether the required fit is to be incresing or decresing Detils The stndrd pool-djcent-violtors lgorithm is used. Miml decresing subsequences re found within the current sequence. Ech such decresing subsequence is replced by constnt sequence with vlue equl to the weighted verge. Within the lgorithm, the subsequence is replced by single point, with weight the sum of the weights within the subsequence. This process is iterted to termintion. The resulting sequence is then unpcked bck to the originl ordering to give the weighted lest squres monotone fit. If incresing=false, the originl sequence is negted nd the resulting estimte negted. The vector giving the best fitting monotone sequence is returned. wmonfrom

8 8 postmen postmen Posterior men estimtor Given dt vlue or vector of dt, find the corresponding posterior men estimte(s) of the underlying signl vlue(s) postmen(, w, = "lplce", = 0.5) w dt vlue or vector of dt the vlue of the probbility tht the signl is nonzero fmily of the nonzero prt of the ; cn be "cuchy" or "lplce" the scle prmeter of the nonzero prt of the if the Lplce is used Note If is sclr, the posterior men E(θ ) where θ is the men of the distribution from which is drwn. If is vector with elements 1,..., n, then the vector returned hs elements E(θ i i ), where ech i hs men θ i, ll with the given. If the qusicuchy is used, the rgument is ignored. If ="lplce", the routine clls postmen.lplce, which finds the posterior men eplicitly, s the product of the posterior probbility tht the prmeter is nonzero nd the posterior men conditionl on not being zero. If ="cuchy", the routine clls postmen.cuchy; in tht cse the posterior men is found by epressing the qusi-cuchy s miture: The men conditionl on the miing prmeter is found nd is then verged over the posterior distribution of the miing prmeter, including the tom of probbility t zero vrince. postmed Emples postmen(c(-2,1,0,-4,8,50), w=0.05, ="cuchy") postmen(c(-2,1,0,-4,8,50), w=0.2, ="lplce", =0.3)

9 postmed 9 postmed Posterior medin estimtor Given dt vlue or vector of dt, find the corresponding posterior medin estimte(s) of the underlying signl vlue(s) postmed(, w, = "lplce", = 0.5) w dt vlue or vector of dt the vlue of the probbility tht the signl is nonzero fmily of the nonzero prt of the ; cn be "cuchy" or "lplce" the scle prmeter of the nonzero prt of the if the Lplce is used Detils The routine clls the relevnt one of the routines postmed.lplce or postmed.cuchy. In the Lplce cse, the posterior medin is found eplicitly, without ny need for the numericl solution of n eqution. In the qusi-cuchy cse, the posterior medin is found by finding the zero, component by component, of the vector function cuchy.medzero. If is sclr, the posterior medin med(θ ) where θ is the men of the distribution from which is drwn. If is vector with elements 1,..., n, then the vector returned hs elements med(θ i i ), where ech i hs men θ i, ll with the given. Note If the qusicuchy is used, the rgument is ignored. The routine clls the pproprte one of postmed.lplce or postmed.cuchy. postmen Emples postmed(c(-2,1,0,-4,8,50), w=0.05, ="cuchy") postmed(c(-2,1,0,-4,8,50), w=0.2, ="lplce", =0.3)

10 10 tfromw tfromw Find threshold from miing weight Given weight or vector of weights (i.e. probbilities tht the prmeter is nonzero), find the corresponding threshold(s) under the specified. tfromw(w, = "lplce", byesfc = FALSE, = 0.5) w byesfc weight or vector of weights specifiction of to be used; cn be "cuchy" or "lplce" specifies whether Byes fctor threshold should be used insted of posterior medin threshold scle fctor if Lplce is used. Ignored if Cuchy is used. Detils The Byes fctor method uses threshold such tht the posterior probbility of zero is ectly hlf if the dt vlue is equl to the threshold. If byesfc is set to FALSE (the defult) then the threshold is tht of the posterior medin function given the dt vlue. The routine crries out binry serch over ech component of n pproprite vector function, using the routine vecbinsolv. For the posterior medin threshold, the function to be zeroed is lplce.threshzero or cuchy.threshzero. For the Byes fctor threshold, the corresponding functions re bet.lplce or bet.cuchy. The vlue or vector of vlues of the estimted threshold(s). wfrom,tfrom,wndfrom Emples tfromw(c(0.05, 0.1)) tfromw(c(0.05, 0.1), ="cuchy", byesfc=true)

11 tfrom 11 tfrom Find threshold from dt Given vector of dt, find the threshold corresponding to the mrginl mimum likelihood choice of weight. tfrom(, = "lplce", byesfc = FALSE, = 0.5) byesfc vector of dt specifiction of to be used; cn be "cuchy" or "lplce" specifies whether Byes fctor threshold should be used insted of posterior medin threshold scle fctor if Lplce is used. Ignored if Cuchy is used. Detils First, the routine wfrom is clled to find the estimted weight. Then the routine tfromw is used to find the threshold. See the documenttion for these routines for more detils. The numericl vlue of the estimted threshold is returned. tfromw, wfrom Emples tfrom(=rnorm(100, c( rep(0,90), rep(5,10))), ="cuchy")

12 12 threshld threshld Threshold dt with hrd or soft thresholding Given dt vlue or vector of dt, threshold the dt t specified vlue, using hrd or soft thresholding threshld(, t, hrd = TRUE) t hrd dt vlue or vector of dt vlue of threshold to be used specifies whether hrd or soft thresholding is pplied A vlue or vector of vlues the sme length s, contining the result of the relevnt thresholding rule pplied to. ebyesthresh Emples threshld(-5:5, 1.4, FALSE)

13 vecbinsolv 13 vecbinsolv Solve systems of nonliner equtions bsed on monotonic function Solve nonliner eqution or vector of nonliner equtions bsed on n incresing function in specified intervl vecbinsolv(zf, fun, tlo, thi, nits = 30,... ) zf the right hnd side of the eqution(s) to be solved fun n incresing function of sclr rgument, or vector of such functions tlo lower limit of intervl over which the solution is sought thi upper limit of intervl over which the solution is sought nits number of binry subdivisions crried out... dditionl rguments to the function fun Detils If fun is sclr monotone function, the routine finds vector t the sme length s zf such tht, component-wise, fun(t) = zf, where this is possible within the intervl (tlo,thi). The relevnt vlue returned is the nerer etreme to the solution if there is no solution in the specified rnge for ny prticulr component of zf. The routine will lso work if fun is vector of monotone functions, llowing different functions to be considered for different components. The intervl over which the serch is conducted hs to be the sme for ech component. The ccurcy of the solution is determined by the number of binry subdivisions; if nits=30 then the solution(s) will be ccurte to bout 9 orders of mgnitude less thn the length of the originl intervl (tlo, thi).

14 14 wndfrom wndfrom Find weight nd scle fctor from dt if Lplce is used Given vector of dt, find the mrginl mimum likelihood choice of both weight nd scle fctor under the Lplce. wndfrom() vector of dt Detils The prmeters re found by mrginl mimum likelihood. The serch is over weights corresponding to thresholds in the rnge [0, 2 log n], where n is the length of the dt vector. The serch uses nonliner optimiztion routine (optim in R) to minimize the negtive log likelihood function negloglik.lplce. The rnge over which the scle fctor is serched is (0.04, 3). For resons of numericl stbility within the optimiztion, the is prmetrized internlly by the threshold nd the scle prmeter. A list with vlues w The estimted weight The estimted scle fctor wfrom, tfromw Emples wndfrom(rnorm(100, c( rep(0,90), rep(5,10))))

15 wfromt 15 wfromt Miing weight from posterior medin threshold Given threshold vlue, find the miing weight for which this is the threshold of the posterior medin estimtor. If vector of threshold vlues is provided, the vector of corresponding weights is returned. wfromt(tt, = "lplce", = 0.5) tt threshold vlue or vector of vlues specifiction of to be used; cn be "cuchy" or "lplce" scle fctor if Lplce is used. Ignored if Cuchy is used. The numericl vlue or vector of vlues of the corresponding weight is returned. tfromw Emples wfromt( c(2,3,5), ="cuchy" )

16 16 wfrom wfrom Find Empiricl Byes weight from dt Suppose the vector ( 1,..., n ) is such tht i is drwn independently from norml distribution with men θ i nd vrince 1. The distribution of the θ i is miture with probbility 1 w of zero nd probbility w of given symmetric hevy-tiled distribution. This routine finds the mrginl mimum likelihood estimte of the prmeter w. wfrom(, = "lplce", = 0.5) vector of dt specifiction of to be used; cn be "cuchy" or "lplce" scle fctor if Lplce is used. Ignored if Cuchy is used. Detils The weight is found by mrginl mimum likelihood. The serch is over weights corresponding to thresholds in the rnge [0, 2 log n], where n is the length of the dt vector. The serch is by binry serch for solution to the eqution S(w) = 0, where S is the derivtive of the log likelihood. The binry serch is on logrithmic scle in w. If the Lplce is used, the scle prmeter is fied t the vlue given for, nd defults to 0.5 if no vlue is provided. To estimte s well s w by mrginl mimum likelihood, use the routine wndfrom. The numericl vlue of the estimted weight. wndfrom, tfrom, tfromw, wfromt Emples wfrom(=rnorm(100, c( rep(0,90), rep(5,10))), ="cuchy")

17 wmonfrom 17 wmonfrom Find monotone Empiricl Byes weights from dt Given vector of dt, find the mrginl mimum likelihood choice of weight sequence subject to the constrints tht the weights re monotone decresing. wmonfrom(d, = "lplce", = 0.5, tol = 1e-08, mits = 20) d tol mits vector of dt specifiction of the to be used; cn be cuchy or lplce scle prmeter in if ="lplce". Ignored if ="cuchy" bsolute tolernce to within which estimtes re clculted mimum number of weighted lest squres itertions within the clcultion Detils The weights is found by mrginl mimum likelihood. The serch is over weights corresponding to thresholds in the rnge [0, 2 log n], where n is the length of the dt vector. An iterted lest squres monotone regression lgorithm is used to mimize the log likelihood. The weighted lest squres monotone regression routine isotone is used. To turn the weights into thresholds, use the routine tfromw; to process the dt with these thresholds, use the routine threshld. The vector of estimted weights is returned wfrom, isotone

18 18 zetfrom zetfrom Estimtion of prmeter in the weight sequence in the EbyesThresh prdigm Suppose sequence of dt hs underlying men vector with elements θ i. Given the sequence of dt, nd vector of scle fctors cs nd lower limit pilo, this routine finds the mrginl mimum likelihood estimte of the prmeter zet such tht the probbility of θ i being nonzero is of the form medin(pilo, zet*cs, 1). zetfrom(d, cs, pilo = NA, = "lplce", = 0.5) d cs pilo vector of dt vector of scle fctors, of the sme length s the lower limit for the estimted weights. If pilo=na it is clculted ccording to the smple size to be the weight corresponding to the universl threshold 2 log n. specifiction of to be used conditionl on the men being nonzero; cn be cuchy or lplce scle fctor if Lplce is used. Ignored if Cuchy is used. If, on entry, =NA nd ="lplce", then the scle prmeter will lso be estimted by mrginl mimum likelihood. If is not specified then the defult vlue 0.5 will be used. Detils An ect lgorithm is used, bsed on splitting the rnge up for zet into subintervls over which no element of ζ cs crosses either pilo or 1. Within ech of these subintervls, the log likelihood is concve nd its mimum cn be found to rbitrry ccurcy; first the derivtives t ech end of the intervl re checked to see if there is n internl mimum t ll, nd if there is this cn be found by binry serch for zero of the derivtive. Finlly, the mimum of ll the locl mim over these subintervls is found. A list with the following elements zet w cs pilo The vlue of zet tht yields the mrginl mimum likelihood The weights ( probbilities of nonzero) yielded by this vlue of zet The fctors s supplied to the progrm The lower bound on the weight, either s supplied or s clculted internlly

19 zetfrom 19 Note Once the mimizing zet nd corresponding weights hve been found, the thresholds cn be found using the progrm tfromw, nd these cn be used to process the originl dt using the routine threshld. tfromw, threshld, wmonfrom, wfrom

20 Inde Topic internl isotone, 6 vecbinsolv, 12 Topic nonprmetric bet.cuchy, 1 bet.lplce, 2 ebyesthresh, 3 ebyesthresh.wvelet, 5 postmen, 7 postmed, 8 tfromw, 9 tfrom, 10 threshld, 11 wndfrom, 13 wfromt, 14 wfrom, 15 wmonfrom, 16 zetfrom, 17 wfromt, 14, 15 wfrom, 10, 11, 13, 15, 16, 18 wmonfrom, 7, 16, 18 zetfrom, 17 bet.cuchy, 1, 3, 10 bet.lplce, 2, 2, 10 cuchy.medzero (postmed), 8 cuchy.threshzero (tfromw), 9 ebyesthresh, 2, 3, 6 16, 18 ebyesthresh.wvelet, 5 isotone, 6, 16 lplce.threshzero (tfromw), 9 negloglik.lplce (wndfrom), 13 optim, 13 postmen, 7, 9 postmed, 8, 8 tfromw, 9, 11, 13 16, 18 tfrom, 4, 10, 10, 15 threshld, 4, 11, 16, 18 vecbinsolv, 10, 12 wndfrom, 10, 13, 15 20