Quadratic forms and a some matrix computations

Transcription

1 Linear Algebra or Wireless Conications Lectre: 8 Qadratic ors and a soe atri coptations Ove Edors Departent o Electrical and Inoration echnology Lnd University it Stationary points One diension ( d d = Stationary yp points : ( d > Stationary point at inia i: ( d d < aia i: ( Copare to aylor epansion at = : Constant Linear Qadratic d d ( = ( + ( ( ( ( + d + d + d ( ( ( -3-4 Ove Edors -3-4 Ove Edors Stationary points [cont.] N diensions ( Stationary points : grad ( ow abot inia or aia? Let s se the aylor epansion at = : ( ( ( Constant Linear = + a+ ( ( B + Qadratic ow do we igre ot i it a aia or inia based on th one? [B called the essian o (]. = = N = a= grad ( N B = N N = -3-4 Ove Edors 3 he qadratic or A general qadratic or in N variables α, α, F(,,, N = αi, j i j = = A i= j= α, α, can, withot loss o generality, be written as (since i j = j i F (,,, N = ( αi, j + α j, i i j i= j= Let s diagonalize! ( N = ( A+ A = B Orthogonal Syetric! F,,, = B= QΛQ = [ z= Q] = zλz Diagonal (real eig.vales We have decopled the variables by aing a change o bas Ove Edors 4

2 he qadratic or [cont.] he qadratic or [cont.] We conclde that,,given a qadratic or, we can write it as: = B= zλz = λ + λ + + λ F z z z N z = Q and QΛQ the diagonalization o the syetric atri B. Using the new coordinates z we can easily conclde that: Let s contine stdying (liited to two diensions or graphical prposes: p (, = B= zλz = λ + λ F z z z = Q and QΛQ the diagonalization o the syetric atri B. (, F Along z there a ini i λ > ai i λ < and or the entire qadratic or a ini i all λ > ai i all λ < Cobining th with or dcssion on aia and inia at stationary points o (: At a stationary point the eigenvales o the essian o ( at = deterines its type. z z Eigenvectors o B -3-4 Ove Edors Ove Edors 6 he qadratic or [cont.] INIU / POSIIVE DEFINIE AXIU / NEGAIVE DEFINIE all λ > F(, all λ < F (, SADDLE / INDEFINIE F(, λ o dierent sign POS. SEIDEFINIE NEG. SEIDEFINIE all λ ( all λ F(, F, Level crve/srace/vole/... o the qadratic or Again, we retrn to: = B= zλz = λ + λ + + λ F z z z N z = Q and QΛQ the diagonalization o the syetric atri B. A level crve/srace/vole/... o at level C given by: B zλz λ λ λ F = = = z + z + + z = C N I the eigenvales all have the sae sign (positive or negative deinite and th sign coincide witht the sign o C, then th epression describes an ellipsoid in N diensions with its principal aes along the eigenvectors o B (colns o Q and with aes lengths C / λ ro the the center. he eigenvale with the saallest agnitde correspond to the longest a Ove Edors Ove Edors 8

3 An pper and a lower bond Gain o the operation A... and again, we retrn to: F = B N QΛQ the diagonalization o the syetric atri B. he orthonoral colns o Q are denoted d q. hen, F = B= QΛQ N ( QΛ ( Q ( qi = = λ a λ q = ( i a λ Using essentially the sae argentation, we can also show that ( F,,, inλ N Eqality obtained when coincide with the eigenvector corresponding to the largest/sallest eigenvale Ove Edors 9 he operation A will deliver a new vector, possibly o a dierent diension than (i A rectanglar. Let s deine the gain o th operation as Gain = A ( nonzero I A rectanglar eigenvales are not deined and it not directly obvios how to ind bonds on th gain. owever, let s do the ollowing: A A AA Gain = = = Now we have a qadratic or A A, A A syetric and we now (sing reslts ro the previos slide: he pper bond A called the atri or in λ a λ spectral nor o the atri A,, and sally the eigenvales are those o A A. denoted A Ove Edors lti-variate noral dtribtions he PDF o a lti-variate noral dtribtion deined as = π ( ep ( ( n ( det ( = E{} the ean vector and = E{(-(- } the covariance atri. syetric and assed to be positive deinite, hence - ets. Both and - are syetric and we have a nice qadratic or in the eponent: ( ( Soe atri coptations showing p in applications Using the reslts on previos slides we can,or instance, ind inia i and aia o th qadratic or and hence aiize and iniize ( nder dirent constraints, lie =. By diagonalizing (or - we can ind any o the properties o a particlar ltivariate dtribtion Ove Edors -3-4 Ove Edors

4 Partitioned atri inverse he atri inversion lea (or Woodbry s identity he inverse o A B = C D = E F G E= ABD C F = A B G =CA = D CA B All indicated inverses are assed to et. Schr copleent o A. Schr copleent o D. he inverse o the Schr copleent o A: E= ABD C All indicated inverses are assed to et. E = A + F G F =A B G =CA = D CA B -3-4 Ove Edors Ove Edors 4 Partitioned atri inverse (version Eaple: Inverting an etended covariance atri Applying the atri inversion lea to the partitioned atri inverse, we obtain that the inverse o A B = C D A F = + G I I F =A B G = CA = DCA B [ ] All indicated inverses are assed to et. Let = E { } be the covariance atri o the stochastic -vector E.g. a odel o a saple easreent = Frther, asse that we have already calclated the inverse -. Qestion: I we decide to add one ore entry, i.e., etend to +, do we need to recalclate the entire inverse ro scratch? NO, se the inverse o partitioned atrices on = E + E E + = E { } E { } = r + r+ r +, series. { } { } -3-4 Ove Edors Ove Edors 6

5 Eaple: Inverting an etended covariance atri Eaple: Saple covariance atri In any applications we need to estiate covariance atrices ro easred data. + + r+ r+, + r = = ( = + r +, + r + r + + r I I he only inverse appearing in the inal epression the already nown Ove Edors 7 An estiate o the covariance atri or a set o easreents o n (zero-ean variables, collected in easreent vectors v : [ ] vv = vv = Asse that we se the inverse o th covariance atri in an algorith (qite coon and we want to pdate the inverse as soon as we receive another easreent. Can we do th, withot re-calclating the entire inverse each tie? First, epress the relation as a dierence eqation: vv [ ] = vv + vv = = [ ] + v v vv -3-4 Ove Edors 8 Eaple: Saple covariance atri We can se the atri inversion lea to derive the ollowing relation: ( α+ βvv and applied to or dierence eqation we get αβ = vv α + α βv v vv [ ] = vv [ ] + vv = vv [ ] vv vv vv ( + v vv [ ] v [ ] [ ] [ ] Sae prodct, only needs one calclation Ove Edors 9