Solution set Stat 471/Spring 06. Homework 2

Transcription

1 oluo se a 47/prg 06 Homework a Whe he upper ragular elemes are suppressed due o smmer

2 b Le Y Y Y Y A weep o he frs colum o oba: A ˆ b chagg he oao eg ad ec YY

3 weep o he secod colum o oba: Aˆ YY weep o he hrd colum o oba: Aˆ YY Alhough lookg a b ugl s edous o smplf hem acuall s he followg: Y Y Y Y [ ] Y Y Ad Y s he same ˆ ha ca be obaed from he ormal equao ˆ Y ad Y Y Y s ˆ hus he fal resuls of he sweep 3

4 procedure are equvale o he soluos of he ormal equaos Checkg oe b oe s que a edous job ad s omed here due o he lm of space Whe a for some posve a a a a ad a B pluggg hese we ca see he dvdebero problems he hrd weep procedure herefore hs algorhm fals o work as he oher algorhms For eample he algorhm usg he ormal equaos marces cao be vered due o rak defcec hs s he mulcollear problem a mulple lear regresso a Le r ad r be colum vecors ha s r r he r Ad le W w j a mar defed as gve whose elemes w j are weghs of varable varaces for he us such ha var w or var W where s a cosa We ca decompose he mar W such ha B lef mulplg b P we have r W PP mar erms r r P P P Le P B P A ad P ad becomes: r r B A 3 4

5 var P var P P W P P PP P Here de mar herefore uder he assumpo of ormal 3 ca be solved for I where I s ˆ MLE usg he ormal equao hs procedure was show class ad hus omed due o lmed space : A Aˆ A B 4 B subsug back 4 becomes: A Aˆ A B P P ˆ r P P P P ˆ P P W ˆ r W 5 5 s he same as r b Frs oba a mar P such ha PP W ce W s a dagoal mar P ca easl be obaed b akg a square roo of each dagoal eleme he lefmulpl b he verse of P s called A for coveece Coduc QR decomposo o A Fall solve for ˆ from ˆ R Q P b backsolvg hs algorhm s llusraed he followg R eample Alhough hs ma o be he bes wa o mpleme hs algorhm eg we ca use backsolvg sead of verg marces shows ha he obaed soluo s he same as MLE show a W < dagc468row4 44 dagoal mar hs W mar s o see wheher he square mar check s workg W < marcrow < c4789 vecor 5

6 < cbdcc34 desg mar wh oe IV hs s a fuco o oba beaha for weghed leas squares esmao usg QR decomposo lbrarma o use gv fuco for geeraled verse beaqr < fucow { f dmw[]dmw[] sopw s o a square mar else { < dmw[] P < mar0rowcol for : P[] < sqrw[] A < gvp qr < qra Q < qrqqr R < qrrqr beaha < gvrqgvp } beaha } he resul s: beaqrw [] [] 574 [] Whch should be he same as he followg beawls < gvgvwgvw [] [] 574 [] he gve lrr fle coas all he ecessar fucos ad eamples so he fucos were esed o he provded eamples smpl b rug he gve fle Ad was checked wheher he wo algorhms produce equvale resuls lke he followg I do o see ahg furher requred o do hs queso: sourcec:\\chaho\\lrr 6

7 he followg code s used o check wheher boh algorhms produce equvale resuls allequalbeaswp asvecorbeaqrb [] RUE hus boh are cocluded o be equvale 4 Necessar fucos are gve he gve pr fle ad hese fucos were adoped for hs eample Newo s mehod ad Fsherscorg mehod were fases b akg 6 eraos from he al value of 3 whle mple scalg ook as ma as 7 eraos I was also checked wheher he 4 mehods reached mamum ad ever mehod ecep mple scalg dd so Alhough mple scalg mehod forcbl sopped eraos afer mamum 00 he 00 h value s close o he olerace o we ma coclude ha he 4 mehods were more or less successfull esmaed he parameer he R code for hese ad he resulg plos are as follows: sourcec:\\chaho\\pr < c sum0 legh sumsum Newos mehod es < roohea03 scoredl dscoredl leghes [] 6 Fsherscorg es < roohea03 scoredl dscoref leghes [] 6 mple scalg sar from above heaha es3 < roohea03 scoredl dscorebo leghes3 [] 7 7

8 eca mehod es4 < secahea03hea9 scoredl leghes4 [] 9 Checkg wheher he eraos reached mamum RUE f mamum s reached f < dles0sum f[leghes] < e6 [] RUE f < dles0sum f[leghes] < e6 [] RUE f3 < dles30sum f3[leghes3] < e6 [] FALE f3[leghes3] [] e05 f4 < dles40sum f4[leghes4] < e6 [] RUE Necessar for plog supp < seq6legh70 ll < log 0log epsupp supp0 sumlogsupp lld < dlsupp0sum lld < dlsupp0sum a plo parmfrowc3 marc33 mgpc0 plo supp ll peb labloglkelhood plo supp lld peb labscore ableh0 plo supp lld peb labobserved fo 8

9 5 ~ Posso ep { } Posso b leg { } µ Y µ he P Y { µ } ep µ { µ } L r ep µ µ where r µ s a vecor for µ he log lkelhood fuco s: [ µ log µ log ] l r µ ep he score fuco for s: r l µ µ µ [ ep { } ] 9

10 ce E [ ] ep{ } E [ ] 0 Y he Fsher formao for s: I E [ { }] ep herefore he epeced score equals ero he MLE for s obaed b solvg ˆ 0 for ˆ he a epresso for he observed formao for s: J ˆ l [ ep{ ˆ } ] ˆ he MLE ad s sadard error are compued o be ad respecvel usg he followg R code Plos are also show below loglk < fucohea{ < c0 < c0385 mu < ephea sum < 0 for :5 sum < summu[][]logmu[]logfacoral[] reursum } score < fucohea{ < c0 < c0385 mu < ephea sum < 0 for :5 sum < sum[]mu[][][] reursum } se < fucohea { < c0 < c0385 mu < ephea fo < 0 for :5 fo < fo[]^mu[] serr < /fo reurserr } 0

11 supp < seq0leghe4 ll < sco < serr < rep0e4 for :e4 ll[] < loglksupp[] for :e4 sco[] < scoresupp[] for :e4 serr[] < sesupp[] parmfrowc3 plosupplllabhealabloglkmalog lkelhoodpel plosuppscolabhealabscoremascorepel ableh0 plosuppserrlabhealabserrmasadard errorpel MLE for hea supp[whchmall] [] adard Error sesupp[whchmall] [] Fshed readg he chaper