Optimal Transform: The Karhunen-Loeve Transform (KLT)

Transcription

1 Opimal ransform: he Karhunen-Loeve ransform (KL) Reall: We are ineresed in uniary ransforms beause of heir nie properies: energy onservaion, energy ompaion, deorrelaion oivaion: τ (D ransform; assume separable) τ uniary τ τ and are random veor fields (i.e., heir elemens are R.V.s) Uniary ransform preserve he energy: τ τ τ τ I ( ) { Uniary ransforms end o sak ransform energy in he firs few oeffiiens: ode ignore 508

2 he Karhunen-Loeve e ransform (KL) Wha do we mean by an opimal ransform? he opimal ransform paks paks he maimum average energy in a given (speified) number of ransform oeffiiens while ompleely deorrelaing hem opimal in he sense of energy paking aording o an error rierion How do we find suh opimal ransform? Opimaliy measure: mean-square error opimal in he mean-square sense Desirable ransform properies (onsrains on ransform): uniary and separable > X X 508

3 he Karhunen-Loeve ransform (KL) e a u e oe e a s o ( ) Consider he D ase for he derivaion of he opimal p ransform Forward ransform: τ Forward ransform: τ Inverse ransform: τ L weighed sum of basis veors 508

4 he Karhunen-Loeve e ransform (KL) oe: τ uniary τ τ I ; i 0; oherwise i i δ ( i, ) { } orhonormal basis veors Goal: Find uniary ransform (i.e. he veors ) ha will allow for he reonsruion of wih as few oeffiiens as possible for a given mse disorion Le R reonsrued image afer eliminaing some ransform oeffiiens 508

5 he Karhunen-Loeve e ransform (KL) { } { } Assume ha we kep ou of he oeffiiens and ha we replaed he remaining wih some onsans whih are independen of he inpu image R + ; < + onsans (usually zero) + Form error: e R ( ) + 508

6 he Karhunen-Loeve ransform (KL) e a u e oe e a s o ( ) rror energy is: ( ) ( ) { + + i i i i i e e ), ( δ ( ) ( ) + ( ) ( ) + ( ) + ( ) + + { } ( ) + Re 508 +

7 he Karhunen-Loeve e ransform (KL) { } Le us firs find he opimal onsans ha will minimize e e oe: If τ real { } real Also, if image real e e ( ) + { } real + [ ] + Find opimal by aking derivaion + e e

8 he Karhunen-Loeve e ransform (KL) In general,, τ and omple diffiul o apply derivaive mehod (sine no analyi eep a 0) bu we should ge same resul. In fa, [ e ] minimized if Re maimized ( e { } ) ma Q [ ] Q [ ] [ ] os Q[] ( ) Q 44 maimum when Q[ ] Q 43 4 a ; where a real > 0 onsan o be deermined ma a a a 508

9 he Karhunen-Loeve e ransform (KL) ( ) ma a a a ake derivaive of previous equaion wih respe o a and se o 0 ( a) 0 a Bu reall ha [ ] [ ] [ ] 508

10 he Karhunen-Loeve e ransform (KL) [] 0 oe: If (whih is ofen he ase sine we usually remove DC before ransmission), we an se 0 ow, we wan o find he opimal ransform by finding he opimal subsiue he epression for he opimal in erms of ino he original error energy epression: e e ( )( ) ( )( ) [( [])( ( []) )] 508

11 he Karhunen-Loeve e ransform (KL) [ [ e e ] [ ] [ ( )( ) ] [( )( ) ] m m ov {} e e ov{} + basis veors of ransform 508

12 he Karhunen-Loeve e ransform (KL) Le C ov { } mari Le us find he opimal sube o he onsrain ha i ( i, ) δ use Lagrange mulipliers o inorporae his onsrain. oe abou Lagrange mulipliers: Given f() and onsrain g()0 ob f() - λg() (obeive funion o minimize) ake ob 0 and ob 0 λ 508

13 he Karhunen-Loeve e ransform (KL) So our opimizaion problem an be saed as follows: [ e ] e inimize sube o C + λ ob [ ( )] C ob λ Assume ransform real { } real 508

14 he Karhunen-Loeve e ransform (KL) { } If real, minimize by aking derivaive wih respe o ob C C λ 0 λ { } { } { } { } eigenveors of ovariane mari C and λ If omple, similar resul wih replaed by ob { } 0 C λ eigenveors of ovariane mari C 508 Karhunen-Loeve Relaion

15 he Karhunen-Loeve e ransform (KL) Definiion: Karhunen-Loeve ransform given by τ τ KL ransform mari if and only if he olumns of are he omple onugaes eigenveors of he ovariane mari C of τ { } (i.e. he olumns of are sine he eigenveors of C are. Subsiue bak he KL relaion ino : [ e e ] e e e C + λ λ λ τ { }

16 he Karhunen-Loeve e ransform (KL) oe: λ represens energy assoiaed wih basis veor. So, in order o minimize he energy error: ake smalles eigenvalues and heir orresponding eigenveors and ignore he oeffiiens orresponding o hese. Keep he eigenveors and, hus, he assoiaed oeffiiens orresponding o he larges eigenvalues 508

17 he Karhunen-Loeve e ransform (KL) Proedure (KL). Compue he orhonormal eigenveors and eigenvalues λ of he ovariane mari C of inpu. Order he veors in dereasing order of λ (veors orresponding o larges λ firs, ) 3. Keep he firs eigenveors. he omple onugae of hese veors form he olumns of where τ [] oe: Veors are ordered suh ha λ > λ > > λ.,,..., KL used o ompare wih oher more praial bu subopimal ransforms. τ 508

18 he Karhunen-Loeve ransform (KL) e a u e oe e a s o ( ) KL relaion in mari form ; Λ C τ τ λ λ O 0 0 Λ where Diagonal mari λ O 0 508

19 he Karhunen-Loeve e ransform (KL) Oher properies of KL aimal deorrelaion C ov [ ( ) ] [( )( ( ) )] τ τ τ τ {} ( [ ]) [ ] τ m m λ 0 τ C τ Λ 0 O 443 τ Λ λ τ { } KL deorrelaes and oeffiiens are ompleely unorrelaed. 508

20 he Karhunen-Loeve e ransform (KL) For a given, he KL ransform paks he maimum average energy in samples of ompared o all oher uniary KL ransforms Le KL of KL A A τ any oher uniary ransform Le σ A, i A [ ( i) ] [ ( ) ] KL i σ KL, i i λ k σ KL, k λk σ A, k k k he KL ransform minimizes [] epeed number of oeffiiens so ha heir oal energy us eeeds a presribed hreshold. 508

21 he Karhunen-Loeve e ransform (KL) For a fied disorion D, he KL ransform ahieves he minimum rae among all uniary ransforms; i.e., Le KL A A τ KL of any oher uniary ransform R KL R A KL daa dependen and has no fas algorihm mainly used for omparison wih oher more praial bu sub-opimal ransforms. DC used beause i is daa independen and lose in performane o KL of naural images near opimal energy saking. 508