Change Detection: Current State of the Art and Future Directions

Transcription

1 Change Detecton: Current State of the Art and Future Drectons Dapeng Olver Wu Electrcal & Computer Engneerng Unversty of Florda

2 Outlne Motvaton & problem statement Change detecton technques Radometrc adjustment Geometrc adjustment Stochastc modelng and hypothess testng Future drectons Probablstc approach Geometrc approach 2 2

3 What s Change Detecton? Open your eyes wde, fnd 5 dfferences 3

4 What s Change Detecton? Open your eyes wde, fnd 5 dfferences 4

5 Motvatons (1) In medcal dagnoss, change detecton can help detect dseases. Healthy 1 month later: retna dsease? 5 5

6 Motvatons In remote sensng, change detecton can help assessng damage from natural dsaster. Blox before Hurrcane Katrna Blox after Hurrcane Katrna 6 6

7 Motvatons In vdeo survellance, change detecton can help detectng suspcous actvtes (actvty montorng). 7 7

8 Problem Statement Change Detecton 8

9 Techncal Challenges Change detecton s an ll-posed problem snce t s hard to defne changes between mages Need to serve specfc purposes (survellance, dsease dagnoss, etc.); hard to quantfy meanngful changes Need to remove nsgnfcant changes Lghtng varaton Brght under sunshne Dark under cloudy weather Camera moton Should not be regarded as change Changes caused by camera moton are nsgnfcant Detectng changes s challengng! 9

10 Outlne Motvaton & problem statement Change detecton technques Radometrc adjustment Geometrc adjustment Stochastc modelng and hypothess testng Future drectons Probablstc approach Geometrc approach 10 10

11 Typcal Procedure of Change Detecton Image1 Image2 Intensty Adjustment Intensty Adjustment Geometry Adjustment Geometry Adjustment Stochastc modelng & hypothess testng Change Mask 11

12 Radometrc Adjustment Why? Elmnate lghtng varatons 12

13 Radometrc Adjustment Why? (2) Mtgate nose X-ray mage of crcut board corrupted by salt-and-pepper nose Nose reducton wth a 3 3 medan flter 13

14 Radometrc Adjustment How? Hstogram matchng: make two mages have the same hstogram Homomorphc flterng: I( x) I ( x) I ( x) l o ln I( x) ln I l ( x) ln I o ( x) I o ( x) exp{ HPF(ln I( x))} Homomorphc Flterng 14

15 Intensty modelng: Gaussan nose Nose Mtgaton Frame/local spatal averagng I( x) I ( x) I ( x) N( x) Speckle nose salt and pepper nose Wdely exst n coherent magery, such as SAR, ultrasound Pa z a PDF: p( z) Pb z b 0 else l o How to mtgate t? Medan flter f ( x, y) medan ( s, t) S xy g( s, t) 15

16 Medan Flterng Example 16

17 Geometrc Adjustment Why? A.k.a. mage regstraton Camera may move Need to algn mages nto the same coordnate system 17

18 Geometrc Adjustment How? Need Intrnsc & extrnsc camera parameters General steps Feature Detecton Feature Matchng Transformaton Model Estmaton Transformaton 18

19 Geometrc Adjustment Example Input mages: Adjusted Images: 19 19

20 Image1 Stochastc Modelng and Hypothess Testng Image2 Image Dfferencng Stochastc Modelng Hypothess Testng (Change Mask Decson) Labeled Images Change Mask 20

21 Stochastc Modelng Process Model Selecton Analytc form Model Parameters Labeled Images Statstcal Learnng Parameters of the Model 21

22 Hypothess Test Hypotheses H 0 : no change H 1 : change Testng: maxmum lkelhood k arg max k{0,1} p( x H k ) Input Images Change Mask 22 22

23 Outlne Motvaton & problem statement Change detecton technques Radometrc adjustment Geometrc adjustment Stochastc modelng and hypothess testng Future drectons Probablstc approach Geometrc approach 23 23

24 Probablstc Approach Flowchart Image1 Intensty Adjustment Geometry Adjustment Image2 Intensty Adjustment Geometry Adjustment Homomorphc Flterng Optcal Flow Image Dfferencng Classfcaton Wth Hdden Markov Tree Model Change Mask 24

25 Statstc Model One Pxel X Y F X: Class Y: Pxel ntensty F: Feature MAP crteron: xˆ arg max P x f arg max p f x P x Lmtaton: x It does not consder spatal correlaton x 25

26 Statstc Model Multple Pxel x 1 x2 x3 xn f1 f2 f3 MAP crteron: xˆ arg max p f x Px x 1 where x [ x, x,..., x ] N 1 2 Advantage: consder spatal correlaton Lmtaton: complexty s too hgh 2 N possble x,.e., O(2 N ) complexty, f x has 2 possble values. 26 f N N T

27 Hdden Markov Tree Model What s hdden Markov tree (HMT)? Advantages of HMT: Utlzaton of spatal correlaton Can use Vterb algorthm whose complexty s O(N 2 ) x f l 1 l State x Feature f, observed x f l 1 x () f () 27

28 Classfcaton Decson: MAP crteron x f f f f xˆ arg maxp arg maxp x x, P x x, P x, for V ( ) ( ) ( ( )) root x x, x, () How to calculate: Testng Image Tranng Feature Extracton f x,px x, V p ( ) f Belef Propagaton P x x () () P x, x f, P x f, V x P f P, x ( ) ( ),f, V x f ( ) MAP-based Decson Devce xˆ 28

29 29 Tranng G g g g g m m f N m m x f 1 )) ( ), ( ; ( ) ( ) ( P w b g r f ) P( max ) ( 1 ) (,, n N n n f x g g g,, K k k k x x x K x x 1 ) ( P, P 1 ) P( f f ) ( P f x ) ( P f x ), ( P ) ( f x x p

30 Tranng samples: Expermental Results Ground Sky Testng mages: Output of classfer 30

31 Expermental Results (2) Input Images Change Mask 31

32 Geometrc Approach: Moton-based Change Detecton Does moton mean change? Global moton s caused by camera moton Local moton s caused by object moton, whch s useful. Image1 Image2 Intensty Adjustment Intensty Adjustment Moton Segmentaton Change Mask 32

33 33 Moton-based Change Detecton (2) How to defne 2D moton? Translaton Affne moton Blnear moton Projectve mappng x y u x t v y t y b x b b v y a x a a u y c x c y b b x b v y c x c y a x a a u u a a x a y a xy v b b x b y b xy

34 Moton-based Change Detecton (3) Sngle-body moton model Multbody moton model Multbody moton estmaton: estmate M n and M can be obtaned lnearly after embeddng x nto a hgher-dmensonal space Moton segmentaton: Refne moton models f ( x, ) 0 M g( x, M) f ( x, M ) f ( x, M ) f ( x, M ) 0 (for x) Do t recursvely untl t converges. n n M M n 1 34

35 Expermental Results Camera moton Object moton 35

36 Grand Challenges n Image Processng & Computer Vson 36

37 Change Detecton n 3D Space Change detecton n 3D space s mportant for homeland securty and mltary Key component of detecton & classfcaton of movng personnel n DARPA VsBuldng program Challenges: need better understandng & explotaton of physcs and magng modaltes See through walls, usng radar, MMW, X- ray, acoustc, UWB, SAR, neutron, gamma-ray, etc. 37

38 Real-tme 3D Imagng of Interor of Buldng & Underground Structure Crtcal for urban warfare Provde crtcal nformaton for commanders to make tactcal decsons; help assess enemy course of acton Need synergstc efforts from dfferent areas Wall/ground penetratng sensors Mcrowave magng Vson processng 3D mage reconstructon Crcuts 38

39 Super Resoluton n Satellte Imagng Can we mprove the resoluton of current satellte magery by a factor of 10 or even 100? Potental mpact: why ths s mportant? Able to see detals never avalable prevously, e.g., recognze human, car, objects of sze ~1m Partcularly mportant for ntellgence, Department of Defense, and homeland securty Possble solutons Mult-vew mage processng of multple satellte mages New magng technques based on physcs 39

40 Thank you! 40