Idea is to sample from a different distribution that picks points in important regions of the sample space. Want ( ) ( ) ( ) E f X = f x g x dx

Transcription

1 Importace Samplg Used for a umber of purposes: Varace reducto Allows for dffcult dstrbutos to be sampled from. Sestvty aalyss Reusg samples to reduce computatoal burde. Idea s to sample from a dfferet dstrbuto tat pcks pots mportat regos of te sample space. Wat E f X f x g x dx Istead of samplg from desty (or probablty g x, sample from a mass fucto) dstrbuto wt desty (or pmf) ( x ). Sce we are samplg from te wrog dstrbuto we ave to make adjustmets our estmator.

2 g( x) ( x) g( X) E f ( X) ( X ) Eg f X f x g x dx f x x dx Ts suggests te followg estmato sceme ) Sample x,, x ) Calculate wegts 3) Use estmator from ˆ µ w x. ( ) g x x wf ( x) f, IS So stead of a regular average, ts estmator s a wegted average. So pots tat occur more ofte uder ( x ) ta g( x ) get dowwegted ad tose tat occur less ofte get upwegted.

3 Notce tat ˆ µ f, IS s a ubased estmate of Eg f ( X) regardless of wc proposal dstrbuto ( x ) as log as support as g( x ),.e. x as te same g( x ) > 0 mples tat ( x ) > 0 Note tat ( x ) > 0 ca be allowed to occur we g( x ) 0, toug dog ts teds to be effcet (but tere are tmes you wat to do ts). 3

4 Sce ˆ µ f, IS s ubased, te ma dea s to pck a dstrbuto ( x ) tat reduces te varace. ( X) ( X) f X g X f X g X Var E µ f To do ts, we wat ( x ) to look lke f ( x) g( x ),.e. make look lke a costat. f ( x) g( x) ( x) Te optmal ( x ) satsfes x f x g x f x g x dx Note tat ts usually ca t be determed, due to te ormalzg costat. However ts does gve us a motvato for pckg ( x ). 4

5 Example: Mote Carlo Evaluato of a Lkelood Rato (Geetcs Example) Assume tat you ave a mssg data model were X ( Xobs, Xms ). Te te observed data lkelood rato satstfes l ( θ, θ ) 0 ( θ ) ( θ ) 0 pθ ( X ) obs θ 0 obs ( obs, ms ) (, ) L L p X E p X X θ X θ 0 obs pθ X 0 obs Xms Ts ca be estmated by samplg z, p X X calculatg from ( ms obs ) ) f ( z ) 0 ( obs, ) (, ) p X z θ p X z θ obs ) lˆ ( θ, θ ) f ( z ) 0, z Suppose tat you are terested gettg l θ, θ, based o ts Mote Carlo estmate. 0 5

6 Ts ca be doe wt te mportace samplg estmate ( obs, ) (, ) p X z lˆ ( θ, θ0) f ( z ) p X z θ θ obs Ts ca be sow to be a ubased estmator l θ, θ. of 0 Geetcs example: Observed Data Model λ λ λ + λ,,, ~Mult 97,,,, ( Y Y Y Y ) 3 4 ( λ ) gy Y Y + Y Y λ λ + λ Complete Data Model ( X, X, X, X, X ) λ λ λ λ ~ Mult 97,,,,,

7 ( λ ) g X X + X X + X X λ λ λ As see before X4 Y4 ~B Y4, + λ Te complete data lkelood rato satsfes (, λ0 ) Y + X Y + Y λ0 λ0 gyx, λ λ λ g Y X 4 3 Note tat ts mples te mportace sample wegt satsfes w I ts case ˆ (, ) θ (,,, θ, θ ) c Y Y Y3 θ l λ λ as te form 0 (,,,, ) ˆ c Y Y Y3 λ λ λ l ( λ, λ0) f ( z ) λ z z 7

8 As may problems, te desred samplg dstrbuto does t eed to be kow exactly, but oly up to te ormalzg costat (.e. l x cg x ). Importace samplg stll works fe ts case. ) Sample x,, x ) Calculate wegts 3) Use estmator x. from ˆ µ w ( ) l x x ( x) f, IS W w wf wf ( x) were W w. 8

9 Propertes of ts estmator ( x) cg( x) dx l x E w( X) f ( X) f x x dx x f ce f X g l x E [ w] ( x) dx x cg x dx c As ts s a rato estmator, t s o loger ubased, but t s cosstet. I addto, we c s kow, ts estmator s ofte preferred to te ubased oe dscussed last tme as t ofte as a smaller mea square error (to be dscussed later). Effcecy of mportace samplg Effectve sample sze ESS ( ) + var ( w( X) ) 9

10 Sce te wegts are usually oly kow up to te ormalzg costat, var ( w( X )) eeds to be estmated by te coeffcet of varato of te uormalzed wegts cv ( w) ( w w) ( ) w Assume for wat follows tat c (ad ts s kow). We ave two estmators µ wf ( x) (ubased) µ ˆ µ wf ( x) w w (rato) Let Z w( X) f ( X). Te by te delta metod ( ) [ ˆ µ ] ( ) + ( ) E E Z w w cov, Var µ + ( wz) µ ( W) 0

11 I addto, Var ( ˆ µ ) + ( µ Var ( w) Var ( Z) µ Cov ( w, Z) ) For te ubased estmator, [ ] ad Var ( µ ) Var E µ µ Tus MSE ad E [ ] ( µ ) ( ) µ Z Var ( Z ) MSE ˆ µ ˆ µ µ + Var ˆ µ MSE + + ( µ Var ( w) µ Cov ( w, Z) ) O( ) Tus te rato estmate s to be preferred we ( wz) > ( w) µ Cov, µ Var

12 (assumg µ > 0). Tat s we w( X ) ad w( X) f ( X ) are gly correlated. I addto, te formula for Var ( ˆ) (wole buc of steps omtted) ( ) µ mples ( ) Var µ Var f X + Var w X g Rearragg ts gves ( f ( X) ) Varg Var + Var ( w X f X ) ( w( X) ) Oe way of tkg of ts statemet s tat x s samples from te proposal dstrbuto wort ( Var ( w( X) )) te target dstrbuto g( x ). + samples draw from Te ce tg wt ts rule of tumb s tat t does t deped o te fucto beg tegrated.

13 Tus te effectve sample sze s a useful measure of te effcecy of te metod we dfferet fuctos f are vestgated wt a sgle sample. Oe cosequece of ts s tat we wat to keep te coeffcet of varato of te wegts well beaved. Oe way of dog ts s to ave te proposal x be eaver taled ta te dstrbuto target dstrbuto g( x ). Ts wll elp mmze E g( X) g( x) ( X) ( x) g X Eg X x dx wc wll keep Var g X X small. For example, use a t dstrbuto wt moderate degrees of freedom stead of a ormal. 3

14 Margalzato mportace samplg Let g( X, X ) ad (, ) X X be two probablty destes were te support of g s a subset of g X, X > 0 mples te support of (e.g. ( ) ( X, X ) > 0). Te Var (, ) (, ) were g( X ) ad ( ) g X X g X Var X X X X are te respectve margal dstrbutos. E (, X ) X (, X) (, ) ( X, X) g( X, X ) g X X g X X X X dx ( ) ( ) X X X g X g( X X ) dx X X X dx g X X 4

15 Tus Var ( ) g X, X g X, X Var E X X, X X, X g X Var X I addto Var (, ) (, ) g X X g X Var X X X (, X) (, X ) g X E Var X X Te mplcato of ts result s tat were possble, mmze te umber of varables you sample as ts wll crease ESS( ). However te computatoal burde must also be cosdered. 5

16 For example, f Var (, ) (, ) g X X g X Var X X X but te computatoal tme volved samplg ( X ) s 4 tmes te tme volved samplg ( X, X ), samplg over te secod space s to be preferred. Smlarly, Rao-Blackwellzato ca be wt mportace samplg, gvg a estmate of te form µ W (, ) we f X X x g, Wle ts s a cosstet estmate, t may ot ave a smaller varace ta te o-rao- Blackwellzed estmate. However may stuatos t sould. Te followg s a example were t dd elp. 6

17 Estmatg recombato fractos wt pedgree data 4 members 7 (ofouders), f 4 (fouders) 8 markers from cromosome 9 #alleles rages from 6 to 8 4 members top geerato ave o marker data Wat to use pedgree to estmate te dstaces betwee te 8 markers 7

18 A D A D No Recombato Recombato A D A D No Recombato Recombato Ca use te recombato fracto (P[recombato]) as a measure of dstace. 8

19 Data Structure x l data o locus l x,..., M x x m m markers o o x D dsease / trat data o x ( x x x ),...,, m y l aplotype for locus l o y ( y y y ),...,, m D D o Allele formato wt paretal source z l ertace vector for locus l o z ( z z z ),...,, m D o Idcators of weter allele erted came from te gradmoter or gradfater Assume tat z l s a vector of legt. Te a estmate of θ j, te recombato fracto betwee markers j ad j + s (, z, ) θ j I zj j+ assumg tat z s kow. 9

20 Ufortuately, for most data sets, z ca t be determed wt certaty. However t s possble to do smulato ad estmate te recombato fractos usg Mote Carlo EM (MCEM) As part of ts procedure s t ecessary to calculate E zl x, θ va Mote Carlo. Tere are two approaces to dog ts. ) Sample samplg ( ) y from p θ yx by mportace Sample ( ) z from ( (), ) p θ zxy Estmate E zl x, θ by ) Sample samplg m w l W z ( ) y from p θ Estmate E zl x, θ by m () () we l, W yx by mportace z y θ 0

21 Bot approaces estmate exactly te same quatty, but te d approac s muc more effcet. Frst te secod approac as a smaller varace. I addto, calculatg te expected value s actually faster ta samplg te z s. (usually does t work out ts way).