SPIE Aerosense 001 Conference on Signl Processing, Sensor Fusion, nd Trget Recognition X, April 16-0, Orlndo FL. (Minor errors in published version corrected.) A Signl-Level Fusion Model for Imge-Bsed Chnge Detection in DARPA's Dynmic Dtbse System Mrk J. Crlotto Veridin Corportion, 1400 Key Blvd., Suite 100, Arlington VA 09 Abstrct Improving chnge detection performnce (probbility of detection/flse lrm rte) is n importnt gol of DARPA's Dynmic Dtbse (DDB) progrm. We describe novel pproch bsed on fusing the outputs from two complementry imge-bsed chnge detection lgorithms. Both use historicl imgery over the region of interest to construct normlcy models for detecting chnge. Imge level chnge detection (ILCD) segments the set of imges into temporlly co-vrying pixel sets tht re sptilly distributed throughout the imge, nd uses sptil normlcy models constructed over these pixel sets to detect chnge in new imge. Object level chnge detection (OLCD) segments ech imge into set of sptilly compct objects, nd uses temporl normlcy models constructed over objects ssocited over time to detect chnge in the new imge. Becuse of the orthogonl mnner in which ILCD nd OLCD operte in spce-time, flse lrms tend to decorrelte. We develop signl-level sttisticl models to predict the performnce gin (output/input signl to noise rtio) of ech lgorithm individully, nd combined using 'nd' fusion. Experimentl results using synthetic perture rdr (SAR) imges re presented. Fusion gins rnging from slightly greter thn unity in low clutter bckgrounds (e.g., open res) to more thn 0db in complex bckgrounds contining mn-mde objects such s vehicles nd buildings hve been chieved nd re discussed. Key Words: Chnge detection, lgorithm fusion, imge-level fusion, performnce modeling, synthetic perture rdr (SAR). 1. Introduction Multi-sensor fusion hs been pplied in vriety of remote sensing nd reconnissnce pplictions including lnd use/lnd cover clssifiction, terrin nd feture extrction, chnge detection, nd object recognition. In DARPA's Dynmic Dtbse (DDB) progrm (Kessler 1990) fusion techniques re used to improve chnge detection nd object clssifiction performnce. This pper describes new wy to improve chnge detection performnce by fusing the outputs from two different chnge detection lgorithms operting on dt from the sme sensor. We show tht becuse of the orthogonl nture of the chnge detection lgorithms used it is possible to chieve significnt fusion gins by this pproch. After briefly summrizing the two chnge detection lgorithms in Section, signl-level sttisticl models re derived in order to predict the performnce of ech lgorithm individully, nd combined using 'nd' fusion (Section 3). Results from severl experiments over different imge bckgrounds re presented in Section 4. Ares for future work re discussed in Section 5.
. Detecting Chnge in Spce nd Time DDB employs two different lgorithms for detecting chnge. The first, known s object-level chnge detection (OLCD), ws originlly developed under DARPA's Semi-Automted IMINT Processing (SAIP) effort (Welby 1999). OLCD detects nd mintins dtbse of trget-like regions in SAR (Tom et l 000) nd electro-opticl (EO) imgery (Hoogs nd Mundy 000). Regions which pper, dispper or chnge stte re lbeled s possible chnges. Conceptully, OLCD sptilly segments imges into compct trget-like regions, models the behvior of these regions, nd detects chnges in time. The second lgorithm, imge level chnge detection (ILCD) ws developed under the DDB progrm for detecting chnges in complex bckgrounds (Crlotto 1999, 000b). ILCD, which opertes on EO s well s SAR, segments set of reference imges (i.e., imges tht do not contin trgets of interest) into sptilly distributed, temporlly co-vrying pixel sets. Ech pixel set corresponds to unique bckground type (Crlotto 000). Chnges in new imge re detected sptilly by compring the vlues within ech pixel set to the verge over the pixel set. Chnges detected by ILCD nd OLCD re fused by the object level chnge fusion (OLCF) component in DDB (Berlin et l 000). Becuse of the orthogonl mnner in which ILCD nd OLCD operte in spcetime (ILCD is temporl segmenttion followed by sptil normlcy modeling nd detection while OLCD is sptil segmenttion followed by temporl normlcy modeling nd detection), flse lrms tend to decorrelte. OLCF exploits this property to enhnce chnge detection performnce by reducing the flse lrm rte. 3. Performnce Modeling To simplify our nlysis we ssume tht OLCD opertes on pixels insted of objects, nd pproximte the sptil nd temporl normlcy models used by ILCD nd OLCD s simple verges in spce nd time (Figure 1). Figure 1 ILCD nd OLCD normlcy models Consider the pixel in Figure 1. Let b n be the other pixels in the sme co-vrying pixel set s. The error in predicting from the sptil verge of the b n (ILCD prediction error) is
e b 1 N b n (1) where the totl number of pixels in the set is N +1. Let c n be the regions (ssumed here to be single pixel in extent) ssocited with over time. The OLCD prediction error is the error in predicting from the temporl verge of the c n where M is the number of reference imges. The men squred ILCD prediction error is: e c 1 M c. () n E[e b ] E N b n N 1 ( N E[ ] N E[b N n ] + NE[b n ] + (N N)E[b n b m ]) (3) Assuming jointly norml rndom vribles we cn prmeterize the sttistics in terms of the mens nd covrinces s follows: E[ ] + E[b n ] b + b E[b n ] b + b E[b n b m ] + b (4) from which we obtin [ ] 1 ( N ( N + ) N ( b + b ) + N( + ) + (N N)( b + b b )). (5) E e b The signl to noise rtio (SNR) is the signl divided by the noise power: SNR b b b + b + 1 +(1 1 N b N ) ( b ) b + 1 + (1 1 N b N ). (6) If we ssume: b b (7) where 0 1, when N is lrge ( ) b SNR b 1 + 1 N +(1 1 N ) ( b) ( 1 )(1 + 1 N ) ( b) ( ). (8) 1
As the covrince between rndom vribles in the ILCD normlcy model increses (pproches unity) the SNR increses becuse we cn do better nd better job of estimting sptilly from the b n. The OLCD SNR is derived in the sme fshion nd is SNR c c ( c) + 1 + (1 1 M c M ) cc ' ( ) c 1 + 1 M +(1 1 M ) (9) where s bove c c cc '. (10) As the number of imges M increses SNR c ( c ) ( 1 )(1+ 1 M ) ( c) ( ). (11) 1 As the covrince between rndom vribles in the OLCD normlcy model increses (pproches unity) its SNR lso increses becuse it cn do better nd better job of estimting temporlly from the c n. In DDB OLCF fuses ILCD nd OLCD outputs t the object level. ILCD objects re connected regions where the squred prediction error exceeds specified constnt flse lrm rte (CFAR) threshold. OLCD objects re compct regions tht hve chnged bsed on similr CFAR criterion. Here we pproximte the performnce of OLCF by modeling it, s we hve ILCD nd OLCD, t the signl (prediction error) level. The expected vlue of the product of the ILCD nd OLCD prediction error imges is E[e b e c ] E N b M n N M c n E[ ] E[b n ] E[c n ] + E[b n c m ] ( + ) ( b + b ) ( c + c ) + ( bc + b c ) (1) from which we derive the OLCF SNR SNR bc b c + b c b c + bc b c + b c ( 1 + ) (13)
where bc. This is nlogous to n 'nd' fusion rule. Tble 1 summrizes the lgorithm correltions: is the correltion of rndom vribles between ILCD nd OLCD normlcy models, nd nd re the correltions within the ILCD nd OLCD normlcy models respectively. Tble 1 ILCD, OLCD, nd OLCF prmeters b b' c c' 1 b 1 b' 1 c 1 c' 1 To show the benefit of fusion we define the OLCF to ILCD, nd OLCF to OLCD processing gins: Gin bc / b 1 1 + Gin bc / c 1 1 + (14) ssuming. In generl, ILCD performs well when is lrge, OLCD performs well when is b c lrge, nd OLCF performs well when is smll. Figure plots theoreticl fusion gins for. Appendix A describes sttisticl methods for estimting,, nd from imges. 50.00 db 40.00 30.00 0.00 bet 0.3 bet 0.4 bet 0.5 10.00 0.00 0.10 0.0 0.30 0.40 0.50 0.60 Figure Fusion gins predicted by signl level performnce model lph
SNR Mesured SNR (db) Tble Trgets in open Mesured Processing Gin (db) Predicted Processing Gin (db) Estimted Component Prmeters Input 43.1 ILCD 4.3-0.8.8 ˆ 0.34 OLCD 3.5-19.6 0 ˆ 0.5 OLCF 44.3 1. 6.5 ˆ 0.06 SNR Mesured SNR (db) Tble 3 Trgets in trees Mesured Processing Gin (db) Predicted Processing Gin (db) Estimted Component Prmeters Input 35. ILCD 38.7 3.5 4.6 ˆ 0.47 OLCD 4.7-10.5 0.7 ˆ 0.31 OLCF 49.8 14.6 8.7 ˆ 0.14 SNR Mesured SNR (db) Tble 4 Trgets ner fcility Mesured Processing Gin (db) Predicted Processing Gin (db) Estimted Component Prmeters Input 6. ILCD 38.6 1.4 9.5 ˆ 0.69 OLCD 17.5-8.7. ˆ 0.41 OLCF 48.4. 16.1 ˆ 0.7 SNR Mesured SNR (db) Tble 5 Trgets embedded in mnmde clutter Mesured Processing Gin (db) Predicted Processing Gin (db) Estimted Component Prmeters Input 7. ILCD 46.9 19.7 8.5 ˆ 0.66 OLCD 14. -13 4.1 ˆ 0.53 OLCF 56. 9 16.4 ˆ 0.34
4. Experimentl Results The performnce of detection lgorithm is determined by its output signl nd noise probbility distributions p(ysignl ) nd p(ynoise). The probbilities of detection (Pd) nd flse lrm (Pf) depend on the operting point t P d (t) p(y > t signl) P f (t) p(y > tnoise). (15) Algorithm performnce cn lso be expressed in terms of the signl to noise rtio (SNR) SNR ( s n ) ( s + n ). (16) Computed over the new imge (input SNR), the SNR is indictive of the performnce of single imge Neymn-Person (CFAR) detector. Computed over the prediction error imge (output SNR), the SNR is indictive of the CFAR performnce of ILCD or OLCD t the signl (pre-detection) level. The difference between input nd output SNR (in db) is mesure of the performnce gin of the chnge detector.to evlute the performnce models we selected four SAR imges collected by Veridin's DCS sensor over the DDB Eglin Site 1 study re. These imges represent the deployment of trgets in four different clutter environments: in the open, ner trees, ner fcility, nd embedded in mn-mde clutter. Tbles - 5 summrizes the experimentl results. The first column in ech tble shows the input SNR nd the output SNR from ILCD, OLCD, nd OLCF (pre-detection level). The mesured processing gin (second column) is the output minus input SNR. The predicted processing gins (third column) were obtined from Eqs. 8, 11, nd 14 using component prmeters (fourth column) estimted from the imges s discussed in the ppendix. As the complexity of the scene increses the fusion gin increses with the highest gins chieved in mnmde bckgrounds. Although the predicted nd mesured performnce gins gree in terms of their overll trend there re significnt differences between them. In heterogeneous scenes, using single globl sttistic to represent the covrinces within different bckground types is one potentil source of error. Another is ttempting to compute sttistics over bckground typescontining limited number of smples (i.e., when M nd/or N is smll). 5. Future Work Improving the ccurcy of our performnce model is one importnt re for future work. In prticulr we shll explore the possibility of developing locl estimtes of performnce over ech bckground type. Becuse OLCD fuses ILCD nd OLCD outputs t the object level, in order to obtin more meningful mesure of performnce it is necessry to model the performnce of the components following CFAR detection. This is second re for future work. Detecting chnge in high clutter bckgrounds with significnt trget obscurtion is n importnt opertionl chllenge. Combining detections cross looks t the signl level using 'or' fusion (Crlotto 000b) or using dptive fusion strtegies (Liggins nd Nebrich 000) re two possible lterntives. Extending our performnce model to other fusion rules is third re for future work.
Appendix - Prmeter Estimtion chrcterizes the performnce of ILCD. To estimte we first compute the correltion coefficient between the new imge y nd its verge over S k i E 1 N y n n y n E 1 N y n n N + N N [ b ( ) ] E y n [ ] n, n S k 1 b N 1 N + 1 1 N when N is lrge. We use the verge of the estimtes over ll pixel sets s the estimted vlue of the imge. (A-1) for chrcterizes the performnce of OLCD. To estimte we mesure the correltion coefficient between pixel in the new imge y nd the temporl verge of the ssocited reference imge pixels x n, nd verge the results to obtin n estimte of for the imge: when M is lrge. E 1 o M x m m 1 c M y m 1 E M x m m M + M M [ c ( ) c c ] E y m [ ] 1 M + 1 1 M (A-) Finlly to obtin n estimte of, which chrcterizes the performnce gin of OLCF, we mesure the correltion coefficient between the ILCD nd OLCD predicted error imges: where f [ ] [ ] Vr e b e c Vr[ e b ]Vr e c (A-3) Vr[ e b e c ] b c + bc Vr[ e b ] b + 1 + (1 1 N b N ) (A-4) Vr[ e c ] c + 1 + (1 1 M c M ) cc' Expnding Eq. 7 we hve
f b b + 1 + (1 1 N b N ) 1 + 1 N 1 + 1 ( ) 1+ 1 M 1 ( ) c + bc c + 1 + (1 1 M c M ) cc' (A-5) from which the following expression is obtined ˆ f 1 + 1 N 1 ( ) 1+ 1 M 1 ( ) (1 ). (A-6) References Mrk Berlin, Mrk Nebrich, nd Mrtin Liggins, "Wide-Are SAR-EO Object Level Chnge Fusion Process for DDB," 000 Meeting of the MSS Ntionl Symposium on Sensor nd Dt Fusion, Kelly AFB, Sn Antonio TX, June 000. Mrk Crlotto, "Imge-level chnge detection," 1999 Meeting of the MSS Ntionl Symposium on Sensor nd Dt Fusion, Applied Physics Lbortory, Johns Hopkins University, Lurel MD, My 1999. Mrk Crlotto, "Detecting prtilly-obscured objects using multi-look SAR," 000 Meeting of the MSS Ntionl Symposium on Sensor nd Dt Fusion, Kelly AFB, Sn Antonio TX, June 000. Mrk Crlotto, "Nonliner bckground estimtion nd chnge detection for wide re serch," Opticl Engineering, Vol. 39, No. 5, My 000. Anthony Hoogs nd Joseph L. Mundy, " Informtion Fusion for EO Object Detection nd Delinetion," 000 Meeting of the MSS Ntionl Symposium on Sensor nd Dt Fusion, Kelly AFB, Sn Antonio TX, June 000. Mrtin E. Liggins II nd Mrk A. Nebrich, "Adptive multi-imge decision fusion," SPIE Aerosense, April 000, Orlndo FL. Otto Kessler, http://web-ext.drp.mil/tto/progrms/ddb.html, July 000. Victor Tom, Helen Webb, nd Dvid Lefebvre, " Wide-Are SAR Object Level Chnge Detection," 000 Meeting of the MSS Ntionl Symposium on Sensor nd Dt Fusion, Kelly AFB, Sn Antonio TX, June 000. Steven Welby, http://www.drp.mil/iso/saip_briefing/index.htm, 1999.