Estimation Theory Notes: Set1

Transcription

1 . Lner tmton Mot mportnt ecton (from premble).,.4,.6 nd.a tmton heor ote: Set. orml quton.. Affne tmton Conder zero men ector lued rndom rble nd. p, q column ector A lner (ffne) etmtor would be formed p q nd b p ˆ b But f both nd re zero men ˆ B electng zero men, we moe from n Affne proce to lner proce remember tte pce lner tem. We could be etmte the tte lue bed on oberton... Men-Squre-rror Crteron Determne the MMO oluton for otng tht nd mn * * mn p * Where tht the ze of q, * q, nd p q hen, ech mnmze one term of. nd mnmzng the nddul lue for effectel mnmze wth repect to the mtr! ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

2 ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB: o form the epected mnmzton proce: (ote q nd q ) o tte th cot functon ote tht e hu the th cot functon element dependent upon the frt nd econd moment of () nd. otce tht the reultng lue wll be purel rel for comple nput! o fnd the globl mnmum for, we mut be concerned wth * p But th where mtr theor cn help. f ou ue the trce of the mtr. r r * * ow the um of cot functon element dependent upon the uto nd cro-cornce mtrce of nd. Agn, the reultng lue wll be purel rel for comple nput! ow we he to form the mnmzton nd ole for nd.

3 ..3 Mnmzton b Dfferentton (grdent method) Ung the grdent for comple lue (tng derte wth repect to ech dmenon, Ung the grdent for rel lue (or clr) ote: For the mth behnd tng the grdent/derte ee Append.B. f we collect ll the mnmzed oluton for, ll referred to ( O ), we cn defne the ml mtr: Whch for nertble become p for ˆ Dr. Bzun, nd other, often referred to th the Wener oluton, but the ctul Wener-opf oluton ole deeper, more comple, nd broder problem thn jut th. ee p. 64 emr nd p Dmenonlt note: tht the ze of p, nd re q, q q, p q, nd p q. ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

4 ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB: Mnmzton b Completon-of-Squre Mth trc (gn), epndng jut the centrl mtr D C D C B B A for B C D nd D B A hen umng the correct oluton from we cn ubttute n for the epnded mtr he cot functon then become o mnmze the cot, we mut he

5 he mnmum cot functon re then n n of the form he ummed cot functon for ll weght cn be decrbed r * where r{} trce functon defned the um of the dgonl element. ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

6 ..5 Mnmzton of the rror-cornce Mtr he me oluton cn be dered ung mtr operton. Frt defnng cot mtr (note tht the order of the epected lue chnged). h cot mtr not clr. pndng the centrl mtr * * for Subttutng wth the mnmum * From whch the clr mme become * * * * * r r ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

7 heorem.. Optml Lner tmton For {, } zero men r.. the lner let-men-qure etmtor (l.l.m..e.) of gen ˆ where n oluton to the lner tem of equton h etmtor mnmze the followng two error meure mn * * nd mn he cot functon on the left the trce of the cot functon on the rght. he reultng mnmum men-qure error re gen b mn mn * r r mn * p ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

8 . Degn mple See on-lne tetboo Chpter 4 Orthogonlt Prncple. ote: thee re ml lner let men qure etmtor (l.l.m..e) hoe n chpter were ml mnmum men qure etmtor (m.m..e.). ht wh oluton dffer [preou emple re repeted,.. nd.. wth epnon of & nd..3. ote the equence ngle oberton, two oberton nd oberton. mple.. (o meurement of bnr gnl). [ee.3.] Bnr lue for trnmon te on the lue of +/- wth dered probblt of ½ (not lw true, but uull dered). f we me oberton/etmte n zero-men, unt rnce Gun enronment: where nd the dtrbuton of re ndependent of ech other. Ung heorem.. where nd Snce nd re zero men nd ndependent Alo hen mn mn nd ˆ * r r * r 4 ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

9 ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB: mple..b (Multple meurement of bnr gnl). [..4.3], ˆ h h ˆ he ml lner let men qure etmtor (l.l.m..e) Defnng the ector nd mtrce et hen

10 ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB: nd ˆ r r mn * mn * r

11 ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB: mple.. (Multple meurement of bnr gnl). for = to - B npecton from preou nwer ˆ mn * r Bref dcuon. multple enor wth ndependent noe Mth rc For,,,, col For the emple problem boe nd he ml lner let men qure etmtor (l.l.m..e) ˆ nd r r mn * * mn mn *

12 ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB: mple..3 (rnmon oer no chnnel). [..4.] h the concept of no communcton chnnel, but wth er mple F chnnel repone. Chnnel modfed mbol z Where.5 z he meured mple mbol z.5 We wnt to etmte two mbol () nd () bed on obered meurement () nd (). z.5 z Comment: th ume the trt of trnmon uch tht (-) = he ml lner let men qure etmtor (l.l.m..e) Defnng the ector nd mtrce

13 et hen ˆ ˆ And mn * r r mn * r * mn r ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

14 mn * r mn * r mn * ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

15 mple..4 (lner chnnel equlzton) - t occurrence h ndcte of: chnnel mprment, mbol cro-tl All rndom procee mut be umed to be WSS (wde-ene ttonr) trnent re gone, tttcl/probbltc chrctertc re ld. Smbol et he meured mple mbol z.5 Where () our norml rndom Gun noe term An equlzer contng of fed, three-tp F flter wll be emploed. For the moment we wll he no tme del ˆ Soluton ˆ ˆ Determne the cornce nd cro cornce Buldng up from () r ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

16 r r r.5. 5 r r et Buldng up ote ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

17 hen, det he mme for the lner tem h perform better thn the preou emple! ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

18 .3 tence of Soluton he oluton to the problem hown to be Under wht condton doe oluton et? And, the oluton unque? h depend on the rn of the uto-cornce mtr of the mpled dt or. We lred now tht non-negte defnte. t wll be non-ngulr when t pote defnte! tence requre t to be pote defnte mtr. he proof n the boo counter proof demontrtng counter rgument to thoe boe: () f the mtr ngulr nfntel mn oluton re poble. () here re nfntel mn oluton f nd onl f the mtr ngulr. A n odd ddton to th proce f the mtr ngulr here re nfntel mn oluton but All the oluton reult n the me lue for the etmtor nd the me lue cot functon! ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

19 .4 Orthogonlt Prncple he lner let-men-qure etmtor dmt n mportnt geometrc nterpretton n the form of n orthogonlt condton. or equlentl but th ˆ herefore, we he the orthogonlt We thu conclude tht for lner let-men-qure etmton, the etmton error orthogonl to the dt nd, n fct, to n lner trnformton of the dt,, A for n mtr A. h fct men tht no further lner trnformton of cn etrct ddtonl nformton bout n order to further reduce the error cornce mtr. Moreoer, nce the etmtor ˆ telf lner functon of, we obtn, pecl ce, tht ˆ ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

20 .5 onzero Men Vrble he dcuon h focued o fr on zero-men rndom rble z nd. When the men re nonzero, we hould ee n unbed etmtor for of the form ˆ b for ome mtr nd ome ector b. A before, the ml lue for {, b} re determned through the mnmzton of the men-qure error, mn, b * where ˆ o ole th problem, we trt b notng tht nce the etmtor hould be unbed we mut enforce ˆ. ng epectton of both de of how tht the ector b mut tf Ung th epreon for b, we cn ubttute ˆ b b b ˆ or ˆ h epreon how tht the dered gn mtr hould mp the now zero-men rble to nother zero-men rble ˆ. n other word, we re reduced to olng the problem of etmtng the zero-men rndom rble from the lo zero-men rndom rble. We lred now tht the oluton found b olng n term of the cornce nd cro-cornce mtrce of the zero men rndom rble. Or n th ce, nd he ml oluton then gen b ˆ ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

21 Comprng th equton to the zero-men ce from hm.., we ee tht the oluton to he nonzero-men ce mpl mount to replcng nd b the centered rble nd, repectel, nd then olng lner etmton problem wth thee centered (zeromen) rble. For th reon, there no lo of generlt, for lner etmton purpoe, to ume tht ll rndom rble re zero-men; the reult for the nonzero-men ce cn be deduced centerng. For computton of the let men qure etmte mnmum men qure error t follow th or equlentl o tht the orthogonlt condton n the nonzero-men ce become or where ˆ ˆ Moreoer the reultng m.m..e. mtr become wth ote: thu wh we he referred to the mtrce uto- nd cro-cornce nted of utond cro-correlton mtrce. ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

22 .6 Lner Model (on-lne Chp. 5) h ubtle chnge from to but when delng wth ector nd mtrce, eerthng mut be reewed. heorem.6. Lner etmtor for lner model: For {,, } zero men r.. tht re relted lnerl For nd uncorrelted wth nertble cornce mtrce. he lner let men-qure (llm) etmtor cn be eluted ˆ or or equlentl ˆ or he reultng mnmum men-qure error mtr become ote: For ndependent, nd ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

23 ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB: And: Proof of the t erton ˆ ˆ Another mtr dentt A D B A D C B A A D C B A Contnung ˆ ˆ ˆ ˆ ˆ ˆ ˆ Wh? f mple (clr), column ector nd ector he orgnl oluton requred mtrc neron the lter neron doe not!

24 he error mtr trnlton conert to form of nd Agn recognzng the form of the mthemtcl coneron A A B Don t ou loe mplfng the mth?! C D A B D A A B C D ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

25 emr : on Zero Men For {,, } non-zero men r.. tht re relted lnerl For nd uncorrelted wth nertble cornce mtrce. he lner let men-qure (llm) etmtor cn be eluted ˆ or ˆ he reultng mnmum men-qure error mtr ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

26 .7. Chnnel tmton (mple Applcton) here re ome pplcton where we wnt to etmte the chnnel repone compred to equlzng the mbol output of the chnnel. he obered dt the receer nput. he chnnel model n F flter of unnown coeffcent. herefore, c n our ce, the nown lue re the trnmtted mbol n tme mple. herefore we he nd we wnt to form ˆ or cˆ c c he problem et-up tht got u here defned n the tet : he chnnel umed ntll t ret (.e., no ntl condton n t del element) nd nown nput equence { ( ) }, lo clled trnng equence, ppled to the chnnel. he reultng output equence { z ( ) } meured n the preence of ddte noe, ( ), hown n the fgure. cc cc ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

27 he quntte {, } o defned re both lble to the degner, n ddton to the cornce mtrce { cc c c, } (b umpton). n prtculr, f the noe equence { ( ) } umed whte wth rnce, then. Wth th nformton, we cn etmte the chnnel follow. Snce we he lner model reltng to c, ndcted b the equton, then ccordng to hm.., the ml lner etmtor for c cn be obtned from ether epreon: cˆ cc cc cc ote for communcton: () Almot eer communcton tem trnmt premble before the ret of the mege ent. he premble compoed of numerou nown gnl egment f not ll nown gnl egment. herefore, chnnel etmton cn be performed nd then ued prt of proceng lgorthm C(z) -. () he length of the chnnel etmton F flter mut be determned for tem nd to perform the computton. Good new/bd new. ou cn lw pc n F length longer thn requred. h m men ou re performng more proceng thn requred, but longer fed length F m be eer to del wth thn one who length mut be etmted nd then computed..7. Bloc Dt tmton A reformulton of Chnnel tmton.7.3 Lner Chnnel qulzton A reformulton of mple..4 Lner Chnnel qulzton.7.4 Multple Antenn eceer A reformulton of mple.. Multple Meurement of Bnr Sgnl ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

28 App.A nge Spce nd ullpce of Mtrce See p. 3 of tet ull pce ector re orthogonl to rnge pce ector.. mportnt n multple ntenn receer pplcton, prtculrl for multple nput, multple output dpte receer (utlzton of mrt ntenn). App.C lmn Flter See p. 8 of tet h Chpter 7 of the on-lne tetboo. ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

29 Problem.,.,.5 Problem. (Mtched Flter Bound): he problem etup follow: we re bcll performng one hot trnmon, where zero men rndom mbol trnmtted oer n F chnnel wth n mpule repone,,,. he receer gnl defned Our objecte to degn n F receer wth mpule repone, bcll mtched flter o to mmze. () he frt t to compute the rnce of the gnl component t the output of the receer. f the receer h tp, then t output gnl wll he gnl nd noe component obtned b conolng the recee flter wth the chnnel nd the noe ector. Ung th reult the rnce of the gnl component cn be computed. (b) ow we hft to rnce of the noe component t the output of the receer. Agn ung the equton obtned n (), we cn compute rnce. Snce we now tht the chnnel noe ector h rnce, the rnce of the noe component wll be equl to. (c) We re now requred to erf the mtched flter oluton,.e., dentf the condton tht mmze the of the receer t tme. he t the output of the receer gen b hen ung Cuch Schwrtz neqult we cn obtn the mtched flter oluton. (d) ere we clculte the lrget lue tht the cn ttn; the mtched flter bound. We lred he obtned n (c). (e) Fnll ung the mtched flter oluton we elute the output of the receer t tme. ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

30 Problem. (Frequenc Domn qulzton): h problem epln the concept behnd frequenc domn equlzton. he dt trnmtted wth cclc prefng oer chnnel wth trnfer functon A bloc of no meurement collected, t dcrete fourer trnform (DF) clculted, then fter clng the reult b n pproprte fctor the etmtor n the frequenc domn cn be determned. h problem ue the reult of Prob..9, more pecfcll the fct tht ung cclc prefng reult n crculnt chnnel mtr. () he frt ecton of the problem to determne the no oberton n the frequenc domn. f the DF mtr of ze, then Λ where,,. he trc here to proe untr, o tht we cn wrte Λ the dgonl mtr wth egenlue denoted b. h reult obtned from the fct tht crculnt mtr cn be dgonlzed b DF. he tetboo prode th reult but fl to epln the concept. We re lo requred to fnd the cornce mtrce of, though th reltel mple. (b) We wll now determne the let men qure etmtor n the frequenc domn. he cornce mtrce, cn be eluted el nd the etmtor determned. he lt prt of th problem to proe h ecton ue the reult tht Λ,,, whch mentoned before h neer been dered. ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB:

31 Problem.5 (ndom elegrph Sgnl): n th problem we model rndom telegrph gnl Poon proce nd then elute rou probblte, cornce mtrce nd then etblh the lner let men qure etmtor. rndom proce wth ntl lue wth probblt /. After ech occurrence of n eent n the Poon proce, chnge polrt. he PDF of the Poon proce gen b,,,.! the erge number of eent per econd, the number of eent occurrng n the nterl,. () Clculte the probblte nd. t ueful to recll the lor ere epnon. Snce,! we now tht for eer een, lue of. So. Smlr rgument cn be mde for. (b) Clculte the probblte nd. We cn ue the reult from () to compute the probblte. (c) Clculte the probblte nd. Ung condtonl probblte we cn etblh tht (d) We cn el how tht zero men. he uto correlton cn be computed b. Snce ndependent of tme nd men zero, lo ttonr. (e) ow let compute the let men qure etmtor of gen both nd. Ung the reult of (d) we cn elute, nd thereb the m.m..e. ote nd fgure re bed on or ten from mterl n the tetboo: Fundmentl of Adpte Flterng b Al.. Sed; Wle & Son, oboen,, 3, SB: