Chapter 4 Linear Models

Transcription

1 Chapter 4 Linear Models

2 General Linear Model Recall signal + WG case: x[n] s[n;] + w[n] x s( + w Here, dependence on is general ow we consider a special case: Linear Observations : s( H + b known observation matrix ( p p known offset (p he General Linear Model: x H + b + w Data Vector Known Fll Rank o Be Estimated Known ~(,C zero-mean, Gassian, C is pos. def. ote: Gassian is part of the Linear Model

3 eed For Fll-Rank H Matrix ote: We mst assme H is fll rank Q: Why? A: If not, the estimation problem is ill-posed given vector s there are mltiple vectors that give s: If H is not fll rank hen for any s :, sch that s H H 3

4 Importance of he Linear Model here are several reasons:. Some applications admit this model. onlinear models can sometimes be linearized 3. Finding Optimal Estimator is Easy ˆ MVU ( H C H H C ( x b as we ll see!!! 4

5 MVUE for Linear Model heorem: he MVUE for the General Linear Model and its covariance (i.e. its accracy performance are given by: ˆ MVU ( H C H H C ( x b ˆ ( H C C H and achieves the CRLB. Proof: We ll do this for the b case bt it can easily be done for the more general case. First we have that x~(h,c becase: E{x} E{H + w} H + Ε{w} H cov{x} E{(x H (x H } E{w w } C 5

6 Recalling CRLB heorem Look at the partial of LLF: ln p(x; [ ] ( x H C ( x H x$!#!" C x $!#!" x C H + $ H!!#!" C! H Constant w.r.t. Linear w.r.t. (H C - x [(H C - x] x C - (H Qadratic w.r.t. (ote: H C - H is symmetric ow se reslts in Gradients and Derivatives posted on BB: ln p(x; [ ] [ ] H C x + H C H H C x H C H ( H$!#!" C H H C H H C x $!!!#!!! " I( g( ˆ Pll ot H C - H he CRLB heorem says that if we have this form we have fond the MVU and it achieves the CRLB of I - (!! 6

7 Whitening Filter Viewpoint Assme C is positive definite (necessary for C - to exist For simplicity assme b hs, from (A.: for pos. def. C invertible matrix D, s.t. C - D D C D - (D - ransform data x sing matrix D: E ~ x Dx DH + { } { } { } ww ~ ~ E ( Dw( Dw E Dww D Dw ~ H + w ~ Claim: White!! DCD D D ( D D I x D Whitening Filter ~ x MVUE for Lin. Model w/ White oise ˆ 7

8 8 Ex. 4.: Crve Fitting Cation: he Linear in Linear Model does not come from fitting straight lines to data It is more general than that!! n x[n] Data Model is Qadratic in Index n Bt Model is Linear in Parameters w H x + 3 H ] [ ] [ 3 n w n n n x Linear in s

9 Ex. 4.: Forier Analysis (not most general Data Model: x[ n] M M πkn πkn ak cos + bk sin + k k Parameters to Estimate w [ n] AWG Parameters: [a... a M b... b M ] (Forier Coefficients Observation Matrix: H cos πkn πkn sin n,,,, Down each colmn k,,, M 9

10 ow apply MVUE heorem for Linear Model: ˆ MVU ( H H H x I Using standard orthogonality of sinsoids (see book ˆ MVU H x Each Forier coefficient estimate is fond by the inner prodct of a colmn of H with the data vector x Interesting!!! Forier Coefficients for signal + AWG are MVU estimates of the Forier Coefficients of the noise-free signal COMME: Modeling and Estimation (are Intertwined Sometimes the parameters have some physical significance (e.g. delay of a radar signal. Bt sometimes parameters are part of non-physical assmed model (e.g. Forier Forier Coefficients for signal + AGW are MVU estimates of the Forier Coefficients of the noise-free signal

11 Ex. 4.3: System Identification [n] H(z + x[n] Known Inpt Unknown System w[n] Observed oisy Otpt Goal: Determine a model for the system Some Application Areas: Wireless Commnications (identify eqalize mltipath Geophysical Sensing (oil exploration Speakerphone (echo cancellation In many applications: assme that the system is FIR (length p x[ n] p k h[ k] [ n k] + w [ n] nknown, bt here we ll assme known Measred Estimation Parameters Known Inpt Assme [n], n < AWG

12 Write FIR convoltion in matrix form: w x H + $!#!"!!!!!!!!!!!! "!!!!!!!!!!! $! # ] [ [] [] ] [ ] [ [] [] [] [] [] [] [] [] ( p h h h p xp Estimate his Measred Data Known Inpt Signal Matrix he heorem for the Linear Model says: ( x H H H MVU ˆ ( ˆ H H C σ and achieves the CRLB.

13 Q: What signal [n] is best to se? A: he [n] that gives the smallest estimated variances!! Book shows: Choosing [n] s.t. H H is diagonal will minimize variance Choose [n] to be psedo-random noise (PR [n] is to all its shifts [n m] Proof ses: ( σ H H C ˆ And Cachy-Schwarz Ineqality (same as Schwarz Ineq. ote: PR has approximately flat spectrm So from a freqency-domain view a PR signal eqally probes at all freqencies 3