Template matching. s[x,y] t[x,y] Problem: locate an object, described by a template t[x,y], in the image s[x,y] Example

Transcription

1 Template matchig Problem: locate a object, described by a template t[x,y], i the image s[x,y] Example t[x,y] s[x,y] Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 1

2 Template matchig (cot.) Search for the best match by miimizig mea-squared error x= y= E p,q = s[x,y] t x p, y q x= y= x= y= = s[x,y] + t[x,y] Equivaletly, maximize area correlatio r[ p,q]= s[x,y] x= y= Area correlatio is equivalet to covolutio of image s[x,y] with impulse respose t[-x,-y] x= y= s[x,y] t[ x p, y q]= s[ p,q] t [ p, q] t x p, y q Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig

3 Template matchig (cot.) From Cauchy-Schwarz iequality r p,q = s x, y t x p, y q s x, y x= y= x= y= x= y= t x, y Equality, iff s x, y = α t x p, y q with α 0 Block diagram of template matcher s[ x, y] t [ x, y ] r x, y Search peak(s) [ ] object locatio(s) p,q Remove mea before template matchig to avoid bias towards bright image areas Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 3

4 Template matchig example x 10 7 t[x,y] s[x,y] r[p,q] Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 4

5 Cosider sigal detectio problem Sigal model s[ x, y] g [ x, y ] r [ x, y ] shifted template Matched filterig Search peak(s) s[ x, y] = t[ x p, y q]+ [ x, y] Problem: desig filter g[x,y] to maximize SNR = r p,q { E x, y g x, y } Object locatio(s) p,q correct peak Other objects : "oise" or "clutter" psd Φ ( e jω x,e jω ) y false readigs Vector-matrix formulatio r p,q = g!! H s covariace R = E {!! } H! s =! t +! SNR = r p,q { } E! g H! Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 5

6 Optimum filter has frequecy respose Proof: SNR = = r!" p,q# $ { E!" x, y # $ g!" x, y # $ } = T Matched filterig (cot.) 1/! " GΦ #! $ " Φ 1/ T # $ dω! dω x y - " G Φ dω dω x y Φ 1 dω x max. SNR dω y ( ) = T ( e jω x,e jω ) y Φ ( e jω x,e jω ) y G e jω x,e jω y ( ) G e jω x,e jω y G( e jω x,e jω ) y T ( e jω x,e jω y )dω x dω y Φ G ( e jω x,e jω y )dω x dω y #! Φ dω x dω y. - T $ " G Φ dω x dω y Cauchy-Schwarz iequality, Φ 1 dω x with equality, iff GΦ 1/ = α Φ 1/ T # dω y. $ Vector-matrix formulatio! g = R 1 t! Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 6

7 Matched filterig (cot.) Optimum filter correspods to projectio o! g = R 1 t! Proof: SNR = r p,q { } E! g H!! g H! t! g H R! g = ( 1/! R g ) H R! 1/ t max. SNR ( ) ( 1/! R g ) H 1/! R g = t! H R! 1 t ( ) ( 1/! R g ) H 1/! R g ( ) ( 1/ R! t ) H 1/ R! t ( 1/! R g ) H 1/! R g ( ) Cauchy-Schwarz iequality, ( ) 1/! with equality, iff R! g = α 1/ R t Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 7

8 Matched filterig (cot.) Optimum detectio: prefilterig & template matchig s[ x, y] h[ x, y] t[ x, y] r [ x, y ] h x, y = e jω x,e jω y Φ ( ) Search peak(s) e jω x x+ jω y y dω x dω y object locatio(s) p,q For white oise [x,y], o prefilterig h[x,y] required Low frequecy clutter: highpass prefilter Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 8

9 Matched filterig example Test Image Template Template Matchig Result Matched Filterig Result Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 9

10 Matched filterig example (cot.) Matched Filter Impulse Respose (180 o rotated) / / / 0 / / -/ - -/ 0 0 -/ - - -/ 0 0 / -/ - - -/ Template Clutter Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 10

11 Phase correlatio Efficiet implemetatio employig the Discrete Fourier Trasform s[ x, y] DFT DFT -1 r[ x, y] Peak detectio t[ x, y] DFT H ( e jω x,e jω ) y Phase correlatio H ( e jω x,e jω ) y = 1 S ( e jω x,e jω ) y T ( e jω x,e jω ) y Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 11

12 Origial image Magitude oly Phase oly Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 1

13 Covolutioal eural etworks MIMO covolutio followed by soft threshold oliearity ( activatio fuctio ) Reduce spatial resolutio, e.g. by dilatio + subsamplig ( Max poolig ) Liear projectio followed by soft thresholdig Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 13

14 MIMO covolutio Sigle-iput-sigle-output: f [x,y] ad g [x,y] are arrays of scalar values Multiple-iput-multiple-output covolutio: L N G g x, y = g 1 g G x, y! x, y f x, y = f 1 f F x, y! x, y L N F Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 14

15 Example templates of first covolutioal layer AlexNet, F=3, G=96 Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 15 [Krizhevsky et al., 01]

16 Activatio fuctio Sigmoid ad tah traditioally used ReLU (rectified liear uit) simpler ad improves covergece of traiig Traied bias is added before activatio fuctio to set the best threshold Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 16

17 AlexNet hyperparameters Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 17 I eural etworks ligo, hyperparameters are set by had beforehad. I additio, AlexNet Has > 60M parameters that are optimized by supervised learig. [Krizhevsky et al., 01]

18 AlexNet Image Classificatio Results [Krizhevsky et al., 01] Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 18

19 AlexNet Image-based Retrieval Results Query Most similar images i database [Krizhevsky et al., 01] Digital Image Processig: Berd Girod, Staford Uiversity -- Template Matchig 19