A Stochastic Approximation Iterative Least Squares Estimation Procedure

Transcription

1 Joural of Al Azhar Uvrst-Gaza Natural Sccs, 00, : A Stochastc Appromato Itratv Last Squars Estmato Procdur Shahaz Ezald Abu- Qamar Dpartmt of Appld Statstcs Facult of Ecoomcs ad Admstrato Sccs Al-Azhar Uvrst - Gaza Rcvd /5/00 Accptd 00 Abstract: Lawto ad Slvstr 97 cosdr a fd sampl sz stmato procdur for a olar rgrsso modl. I ths papr w propos a stochastc appromato tratv last squars procdur. Our Procdur lads to a sgfcat rducto th sampl sz.. INTRODUCTION AND SUMMAR Cosdr th followg olar rgrsso modl: g ; P r Whr g:,. wth P ad r bg Euclda spacs, s a uobsrvabl radom rror, wth E = 0, var ; s a costat that ma dpd o ; = s a obsrvabl r radom rspos that ca b obsrvd at ach lvl ; ad P s th paramtrs of trst. Basd o th obsrvatos,,.... t has b kow, [3], [4], [5], ad [7], va classcal procdurs, how to stmat,..., P. Our trst wll b th class of modls whch cota a compot lar som paramtrs but olar th rmag paramtrs. Th obctv

2 h o A Stochastc Appromato Itratv Shahaz Abu- Qamar wll b to stmat squtall usg a tchqu whch th optmal stochastc appromato mthod [], s combd wth th approach of lmatg lar paramtrs proposd b [4]. Th squtal procdur s also compard wth th fd sampl sz procdur basd full o th Lawto ad Slvstr mthod. Now to achv our obctv,.. to stmat squtall, w ca th us th followg optmal stochastc appromato procdur []: Choos as a arbtrar tal stmat of, th df th stmatg squc whr b: a h,,,... I grad f, / f, o., =.. ad a s squc satsfg a, a Fshr formato matr.,. g. a a /, a 0, ad I s P P Th Lawto ad Slvstr mthod 97 of stmato ma b appld wh th olar rgrsso modl. has th spcal form 36

3 whr Joural of Al Azhar Uvrst-Gaza Natural Sccs, 00, q, g ;,,,...,, q.3, tr larl to th modl.3, rprsts th vctor of olar paramtrs.3, ad th g ; ar fuctos ol of th olar paramtrs ad th prdctor varabls,.. p q r g :,,,..., q. thr mthod s usd for th fd sampl sz cas wh obsrvatos,,..., ar avalabl. Usg thr procdur, w tak as a tal valu of ad th dtrm th compao st of "bst" valus for b th ordar last squars procdur. Lt ^,,,,...,, q rprst th vctor of last squars stmats of th s assocatd wth a gv st of ' s; ', aml,,...,,,pq Lt dot th colum vctor of obsrvd rspos valus assocatd wth th obsrvd valus of th prdctor vctor,,,,...,. ' 37

4 A Stochastc Appromato Itratv Shahaz Abu- Qamar Lt G dot th q matr wth lmts g ;,,,..,,,,..., q., provdd that b: It th follows that, th vctor G' G G ' G sts; s gv G' Th rducd "modl" assocatd wth.3 s th gv b: q *, ; g..4 Sc ar strctl fuctos of 's, th modl..4 s a olar rgrsso modl wth ol p-q paramtrs rathr tha th p paramtrs th orgal modl. Lawto ad Slvstr 97 proposd a tratv mthod lk th larzato mthod, ad stpst dsct mthod, [4] to stmat th rmag ukow olar paramtrs. Ths procdur of Lawto ad Slvstr stmats th olar paramtrs a cosqut fasho, that s th whol data must b usd to fd valus of th stmators. If th data s draw squtall, th ths procdurs wll ot b sutabl to us. Howvr th stochastc appromato procdurs hav b show to b "optmal" [] th ss that th stmatg squc s a cosstt ad asmptotcall ffct stmator of such that, th varac of th asmptotc dstrbuto of achvs th Cramr-Rao lowr boud for th varac of a ubasd stmator of. Th abov rsults show that t s worthwhl to cosdr th us of stochastc appromato procdurs to stmat 38

5 Joural of Al Azhar Uvrst-Gaza Natural Sccs, 00, squtall th olar paramtrs.4, stad of usg a tratv classcal mthod. W wll llustrat th fd sampl sz procdur of [4] usg th followg ampl gv b ths authors. Lt.5 Whr ad ar two ukow paramtrs to b stmatd, s a uobsrvabl radom rror ad s a rspos varabl at th lvl. appars larl th modl.5. W sk th last, squars stmators whch mmz Q,.6 It follows that th bst valu of gv dotd b.7 Now substtut.7 to.5. Th lar paramtr s automatcall rplacd b ts bst compao valu fucto of alo. O th obtas th rducd "modl", gv b: whch s a *.8 39

6 A Stochastc Appromato Itratv Shahaz Abu- Qamar Th paramtr wll b stmatd tratvl b usg a of th tratv mthods mtod prvousl scto.. STOCHASTIC APPROIMATION ITERATIVE LEAST SQUARES PROCEDURE I ths scto, w dscrb a w squtal procdur for stmatg th paramtrs th modl gv b.3, whch combs th stochastc appromato tchqu, wth th tratv last squars tchqu. For abbrvato ths wll b rfrrd to, as th SA-ILS procdur. Clarl, th rducd "modl".8 s a olar modl wth a sgl paramtr. I ordr to stmat squtall b usg optmal stochastc appromato procdur s Sc., w shall cosdr crta probablt modls for b usg th rducd "modl".8, ad th fd th probablt dst fucto for, f ; b trasformato. Thus b usg optmal stochastc appromato procdur of th form., ordr to stmat squtall, choos 40 of, ad th df th stmatg squc as a arbtrar tal stmat ^ ; / d ; ^ b: df a I,,,.... f whr a / sc th optmal valu of a that mmzs th varac of th asmptotc dstrbuto of b. For otatoal covc w st f ;, f ; Th. bcoms [] s gv ad dot b.

7 Joural of Al Azhar Uvrst-Gaza Natural Sccs, 00, ; / d ; ^ df I,=,,. f ^ ^ Lt.,...,, q tratv last squars stmats. ' rprst th vctor of Lt b th frst obsrvato, two obsrvatos, ad so o, b th vctor of th frst dotg th vctor of th frst obsrvd rspos valus whch hav assocatd obsrvd valus of th prdctor vctor,,,,...,. Lt G dot th q matr wth lmts g ;,,,...,,,,...,q It th follows that th squc, provdd that G ' G sts, s gv b: G, =,,. G G.. 4

8 A Stochastc Appromato Itratv G G G Shahaz Abu- Qamar =,,.3 Whr s a arbtrar tal valu for th squc, ad G G G s a tal valu for th squc Th rducd modl assocatd wth.3 s th gv b: Sc th q,, * g ;.4 ar strctl fuctos of th ' s; th modl.4 s a olar rgrsso modl wth ol p-q paramtrs, ad w wll stmat thm b usg th optmal stochastc appromato procdur of th form.. Th ma da of SA- ILS procdur s to stmat th paramtrs whch tr th modl larl, b usg a tratv form of last squars stmator, squtall, ad th us a propr optmal stochastc appromato procdur to squtall stmat olar paramtrs. Thrfor, w wll us tratv last squars procdur ordr to stmat squtall. Now, w llustrat th us of th tratv last squars procdur usg th ampl dscussd b [4]. From.7 w hav 4

9 Joural of Al Azhar Uvrst-Gaza Natural Sccs, 00, 43 Gv a tal guss, w th hav: ^ ^ ^ Substtut., w wll gt ad so I gral, at stag, b substtutg., w gt ad th,=,,,.5 Thrfor th SA_ILS procdur s gv b th followg two coscutv procdurs:

10 A Stochastc Appromato Itratv Shahaz Abu- Qamar 44 ^ ; / ; f d df I, =,, ad, =,, Whr s a arbtrar tal valu for th squc ad / s a tal stmat of basd o Th followg algorthm llustrats th computato of th frst thr stmats for ad th prvous ampl usg th SA_ILS procdur: Stp : Italzato: lt b a arbtrar tal stmat of. Stp : Frst appromato: For = ad data,, th valu of whch mmzs s obtad as /

11 Joural of Al Azhar Uvrst-Gaza Natural Sccs, 00, 45 Stp 3: Improvmt: Tratg as f t was th kow tru valu of, th scod stmat of s obtad from, ^ ^ ; / ; f d df I, =,,... Stp 4: For = ^ ad data,,...,,,,, th last squars stmat for s asl s to b th form, =,, Stp 5: Rpat th abov stps utl, whr s a small spcfd postv umbr. 3. EAMPLE OF THE USE OF THE SA- ILS PROCEDURE UNDER DIFFERENT ERROR DISTRIBUTIONS W shall cosdr th followg olar rgrsso fucto; 3. Eampl: [] Lt s

12 A Stochastc Appromato Itratv Shahaz Abu- Qamar Also, assum th followg two probablt modls for : 3. s assumd to b ormall dstrbutd wth ma zro ad varac, 3.3 s assumd to hav a T- dstrbuto wth r dgrs of frdom r Whch cluds th Cauch dstrbuto r =. Frst of all w wll pla a aaltcal form th stps of th procdur for ths sampl. Udr 3. It follows that s also dstrbutd as N S ;. Tratg as kow tall; ad dffrtatg th log of th dst of wth rspct to w gt: d l f ;, d Th Fshr formato, I cos s,, s s to b qual to I 3. cos From Scto th optmal trasformato for th stochastc appromato procdur s d l f ;, h I o d,,,... 46

13 Joural of Al Azhar Uvrst-Gaza Natural Sccs, 00, 47 cos s,... 0,,, ; v v Th optmal valu of a that mmzs th varac of th asmptotc dstrbuto of [] s gv b, thus a Now choos as a arbtrar tal stmat of, th df th stmatg squc b: cos s,,,... 0,,,...},, 0 { v v ad th stmatg squc of ^ s gv b:

14 A Stochastc Appromato Itratv 48 Shahaz Abu- Qamar ˆ ˆ s ˆ ˆ s ˆ, ˆ s =,, ; vπ, v=0,,,, whr =0,,, ˆ ˆ / s, ad vπ, v Udr 3.3 t follows that th dst of s f r / ;, s r / r / r,. r Tratg as kow tall, w obta, aftr som tdous mapulatos, that whr R I 3/ / Rr r cos d, r 3 cos r 0 r / r r r It s straght forward to show that a altratv rprstato for I s I 5/ / r R r cos r 3 r 0 d 3. whch has th advatag of a smallr powr th tgrat. Usg 3., w gt th followg: lt ˆ b a arbtrar tal stmat of, th t follows, aftr som smplfcatos, that th stmatg

15 Joural of Al Azhar Uvrst-Gaza Natural Sccs, 00, squc ˆ s gv b: / ˆ ˆ ˆ r r 3 s cos d 0 ˆ ˆ ˆ r ˆ ˆ ˆ ; cos r s 0 v, v 0,,,... stmatg squc ˆ s gv b: ;,,..., r, ad, th 3.3 ˆ ˆ s ˆ ˆ s ˆ ; s ˆ,,..., v for =,,,, v = 0,,,, whr ˆ /s ˆ. Takg dffrt cass of dgrs of frdom r, th tgral 3.3 ca b show to b π/,, /3, 384/945 for r=,,4,0, rspctvl. Th stmatg squc ˆ ach cas has th form ˆ ˆ ˆ r s ˆ ˆ cos ˆ ˆ ˆ s ˆ r ˆ ˆ ;,,..., 0 v, v 0,,,... whr r 4,5,7,3 for r =,,4,0 rspctvl. I ach cas th stmatg squc ˆ s gv b 3.4.Th abov coffct valus for r suggst that th gral form s r r 3, but w hav ot provd ths aaltcall. 49

16 A Stochastc Appromato Itratv Shahaz Abu- Qamar 4. NUMERICAL SOLUTION USING THE LAWTON AND SLVESTRE PROCEDURE Lawto ad slvstr cosdrd th spcal cas wh th modl has a lar ad olar compot s quato.3. Th troduc a modfcato basd o th da of rducg th umbr of paramtrs that must b stmatd b th tratv mthods. For a sampl,,,, th lar paramtrs ar stmatd at ach stag b ordar last squars ad th stmats ar th substtutd to.3. A gral outl of th procdur was gv s Scto. W wll dscuss ow th prvous ampl som dtal. Eampl: W cosdr th modl gv Scto 3, that s s I Scto w hav foud that th last squars stmat of gv b: ˆ ad, th rducd " modl " wll b s s, ˆ s *, Whr ths "modl", s tratd as a olar modl wth a sgl paramtr. W hav usd a larzato mthod as a tratv mthod for stmatg th olar paramtr. Ths mthod has b plad prvousl [4]. W wll appl t drctl as follows: Lt 50

17 Joural of Al Azhar Uvrst-Gaza Natural Sccs, 00, 5 ; cos ˆ cos ˆ cos ˆ Z ; s ˆ s ˆ s ˆ g J=,,. Th, df a stmatg squc b: ˆ ˆ g Z Z Z,,, Provdd that Z Z sts,.. ˆ ˆ ˆ cos ˆ ˆ ˆ ˆ ˆ cos ˆ ˆ ˆ s =,,, whr ˆ s a tal stmat of. Th abov stmats, ˆ, wll b tratvl computd, ach trato th "bst" compao valu of ˆ ˆ, wll b computd b th last squars mthod. 5- A SIMULATION STUD I ths scto w rport th fdgs of a smulato stud to compar th proprts of th SA-ILS procdur ad Lawto ad Slvstr fd sampl sz procdur. Th modl usd that w dscussd Scto 3 whr s Whr, th rror trm s assumd to hav є N0, ad T- dstrbuto wth

18 A Stochastc Appromato Itratv Shahaz Abu- Qamar r =,0. Dffrt Dsds ar tak as 34.0D0, D0, 7.0D0,.0D0 grat four sampls ach of sz 00 from th T- dstrbuto mtod abov, usg th IMSL rout am GGAMR. Valus of, ar tak as 0.5,0.65, 0.45,0.85,0.55,0.95 ad 0.45,0.95 wr usd to gv dffrt pattr for th stmats ad umbr of obsrvatos dd for covrgc. For th fd sampl sz -000 was usd, w gt th sam rsults as f th sampl sz =00. Th rsduals є wr gratd usg th radom ormal dvat grator avalabl th IMSL rout am GGNML. W ar trstd comparg th SA-ILS procdur ad th fd sampl sz Lawto ad Slvstr procdur from th pot of vw of th umbr of obsrvatos dd for covrgc. Th followg tabls gv th sampl umbrs of obsrvatos dd for covrgc for th Lawto ad slvstr, ad SA-ILS procdur, whr s tak to b 0 -. Tabl 5.: Modl: Dsd, D0.0D0.0D0 s, whr N0, 0.5, , ,0.95 c. Mas o covrgc Ital ˆ SA-ILS Procdur o. of obsrvatos Lawto ad Slvstr Procdur No. of trato s 5 c.* c.* 5

19 Joural of Al Azhar Uvrst-Gaza Natural Sccs, 00, Tabl 5.: Modl: dstrbuto wth d.f.= Dsd 34.0D0 34.0D D D0, 0.5, , , ,0.95 s, whr Ital ˆ SA-ILS Procdur o. of obsrvatos 9 9 T- Lawto ad Slvstr Procdur No. of trato s c.* c.* 08 c.* Tabl 5.3: Modl: dstrbuto wth d.f.=0 Dsd 7.0D0 7.0D0 7.0D0 7.0D0 7.0D0 7.0D0 7.0D0 7.0D0, 0.55, , , , , , , ,0.65 T- s, whr Ital ˆ SA-ILS Procdur o. of obsrvatos Lawto ad Slvstr Procdur o. of obsrvatos 66 0 c.* c.*

20 A Stochastc Appromato Itratv Shahaz Abu- Qamar 6. Dscusso ad coclusos W obsrv from th abov tabls that th SA-ILS procdur s mor supror tha th Lawto ad slvstr procdur, th ss that th SA-ILS procdur rqurs umbrs of obsrvatos ragg from 4 to, whl th Lawto ad Slvstr procdur rqurs ma tratos ach usg all avalabl 00 obsrvatos, th umbr of th tratos s vr larg most cass. Thus th SA-I L S procdur lads to a sgfcat rducto th amout of obsrvatos ad calculatos rqurd. Rfrcs. Abdlhamd, Sam N. O stmato va optmal stochastc appromato procdurs, Statstcal Thor, ad Data Aalss,. Matusta Edtor Elsvr Scc Publshrs B.V. North Hollad Albrt, A. E. ad Gardr, L.A. Stochastc appromato ad olar rgrsso, Rsarch Moograph No.4, M. I.T. Prss, Cogz P.; Pag D.; Burgot G.; Alla H.; Burgot J.-L 995 Elmato of lar paramtrs o-lar rgrsso: a fast ad ffctv mthod for th dtrmato of bdg paramtrs. Itratoal Joural of Pharmacuts, 3, Drapr, N.R. ad Smth, H. Appld Rgrsso Aalss, Joh Wl ad Sos, Ic., Nw ork Huag, M. N. L. ad Huag, M., 99. A Paramtr-Elmato Mthod for Nolar Rgrsso wth Lar Paramtrs ad Autocorrlato Errors Bomtrcal Joural 338, Lawto, W.H. ad Slvstr, E.A. Elmato of lar paramtrs olar rgrsso, Tchomtrcs, 3, ; Dscusso, McCullagh, P. ad Nldr, J.A. Gralzd Lar Modls, chapma- Hall Lodo