A PODS-based Extended Kalman Filter: Quantifying Sensing Uncertainties in Automatic Bird Species Detection

Transcription

1 IEEE ICRA Workshop on Uncertainty in Automation, May 9, 2011 A PODS-based Extended Kalman Filter: Quantifying Sensing Uncertainties in Automatic Bird Species Detection Dezhen Song Associate Professor Dept. of Computer Science and Engineering, Texas A&M University

2 Thanks to: Ni Qin, Yiliang Xu, Chang Young Kim, Wen Li, TAMU Ken Goldberg, UC Berkeley Ron Rohrbach, Cornell Lab of Ornithology John Fitzpatrick, Cornell Lab of Ornithology David Luneau, U Arkansas Hopeng Wang and Jingtain Liu, Nankai University John Rappole, Smithsonian Selma Glasscock, Welder Wildlife Foundation National Science Foundation The Nature Conservancy Arkansas Game and Fish Commission U.S. Fish and Wildlife Service Arkansas Electric Cooperative Cache River National Wildlife Refuge

3 Biological observation is arduous, expensive, dangerous, lonely

4 Assisting the search for IBWO

5 Detecting Rare Birds Low occurrence (e.g., <10 times per year) Short duration (e.g., < 1 sec. in FOV) Huge video data for human identification. Setup and maintenance in remote environments.

6 Design Goals Accuracy low false negative Data reduction filtering the targeted bird Easy to setup and maintain monocular vision system

7 Related Work Natural cameras DeerCam Africa web cams at the Tembe Elephant part Tiger web cams James Reserve Wildlife Observatory Crane Cam Swan Cam

8 Related Work Motion detection and tracking Elgammal, Grimson, Isard Periodic motion detection Culter, Ran, Briassouli 3D inference using monocular vision Ribnick, Hoiem, Saxena

9 Related Work Kalman Filter SLAM, tracking, recognition Convergence ample observation data manageable noise less than 11 data points significant image noise

10 Bird detection problem Input targeted bird body length l b and speed range V=[v min,v max ]. a sequence of n images containing a moving object Output to determine if the object is a bird of targeted species

11 Assumptions Static monocular camera High resolution Narrow FOV Single bird in FOV Motion segmentation Constant bird velocity High flying speed Narrow camera FOV

12 Observation 1: Invariant body length in Steady flight

13 Invariant body length in steady flight [u t,v t ] T [u h,v h ] T z=[u h,v h,u t,v t ] T (observation)

14 Bird Body Axis Filtering Observation 2: Body axis orientation close to tangent line of trajectory during steady flight Flying trajectory Bird body axis B θ θ Difference between θ and θ on 61bird sequences: o µ = 0.8 ; σ = 8.3 b ( u, v ) B t t ( u, v ) B b z= argmax l, s.t. θ [ θ 2 σ, θ + 2 σ ] h h o b b

15 Modeling A Flying Bird [x,y,z] T P tail Kinematics: Tail: x xl & b / v t t t T P tail = [ x, y, z ] = y yl & b / v z zl & b / v [u t,v t ] T [u h,v h ] T camera center z Image plane y x Pin-hole model:

16 Extended Kalman Filter x(k) x(k+1) z(k) z(k+1) Image plane z y camera center x

17 Determine Species for Noise-free Cases False Image plane camera center Targeted range True

18 Estimation with Observation Noises Image plane camera center

19 Probable Observation Data Set (PODS) ( ) [ h S ( ) ] 1 k = u k ± τ S ( ) [ h ( ) ] 2 k = v k ± τ t S3( k) = [ u ( k) ± τ ] S ( ) [ t ( ) ] 4 k = v k ± τ S( k) = S ( k) S ( k) S ( k) S ( k) Image plane camera center Targeted range 1 n 1 n 1 n PODS: Z : = {Z : z( ) S( ) and (X : ) < } k k ε δ

20 EKF Convergence Metrics

21 PODS-EKF Decision-making: I 1: n (Z ) = 1: n 1 (accept) if V V Φ and Z Φ 0 (reject) otherwise PODS: Z = {Z z( k) S( k) and ε(x ) < δ } 1 : n 1 : n 1 : n Targeted range Dezhen Song and Yiliang Yu, Low False Negative Filter for Detecting Rare Bird Species from Short Video Segments using a Probable Observation Data Set-based EKF Method, IEEE Transactions on Image Processing, vol. 19, no. 9, Sept. 2010, pp

22 PODS-EKF Approximate Computation % = arg min ε(x 1: n 1: n Z ) z( k ) S( k ) Subject to: Z = {Z z( k) S( k) and ε (X ) < δ } 1 : n 1 : n 1 : n Targeted range Dezhen Song and Yiliang Yu, Low False Negative Filter for Detecting Rare Bird Species from Short Video Segments using a Probable Observation Data Set-based EKF Method, IEEE Transactions on Image Processing, vol. 19, no. 9, Sept. 2010, pp

23 Dezhen Song and Yiliang Yu, Low False Negative Filter for Detecting Rare Bird Species from Short Video Segments using a Probable Observation Data Set-based EKF Method, IEEE Transactions on Image Processing, vol. 19, no. 9, Sept. 2010, pp

24 Algorithm

25 Experiments Both simulated and real data A desktop PC with an Intel Core 2 Duo 2.13GHz CPU and 2GB RAM Matlab 7.0 (motion detection) and Visual C (PODS-EKF) Arecont AV3100 camera Bird species tested:

26 Convergence of different EFKs on Rock Pigeon

27 Simulation on three birds

28 Physical Experiment on Rock Pigeon Insects, falling leaves, other birds, etc.

29 ROC Curves for Rock Pigeon Area under ROC curve: 91.5% in Simulation; 95.0% in Experiment.

30

31

32

33

34

35

36

37 Data reduction Oct Oct Motion detection: TB to GB PODS-EKF: GB to MB (~960 images) Overall reduction rate: %

38 What we found Pileated woodpecker (cousin of IBWO)

39 Northern flicker (smaller than IBWO)

40 Red-tailed Hawk (larger than IBWO)

41 Conclusion Low false negative bird filter: PODS-EKF Cope with insufficient noisy observation data 95% area under ROC curve % data reduction

42 Current and Future Work Examine wing-flapping motion Wingbeat frequency is unique for each species

43 Wing Kinematic Model

44 Current & Future Work: AnyFish Collaborators: Mr. Ji Zhang, Dr. Gil Rosenthal, and Dr. Wei Yan

45 Thanks! Websites:

46

47 Seagull: Mean 2.74 Hz S.D Hz Gliding component Wingbeat frequency component Harmonic component