Su Liu 1, Alexandros Papakonstantinou 2, Hongjun Wang 1,DemingChen 2

Transcription

1 Real-Time Object Tracking System on FPGAs Su Liu 1, Alexandros Papakonstantinou 2, Hongjun Wang 1,DemingChen 2 1 School of Information Science and Engineering, Shandong University, Jinan, China 2 Electrical & Computer Engineering Department, University of Illinois, Urbana-Champaign

2 Outline Introduction Object Tracking Framework Searching Object Map Techniques Cascade Boundary Joiner Experiment Results

3 Outline Introduction Object Tracking Framework Searching Object Map Techniques Cascade Boundary Joiner Experiment Results

4 Applications of Object Tracking

5 Tasks of Object Tracking Object tracking is used for identifying if i the trajectory of moving objects in video frame sequences. Build the model of what you want to track. Use what you know about the previous frames to make estimation about the position of objects in current frames.

6 Real-time object tracking approaches Object detection Point-detection schemes : find interest points in images which have an expressive texture in their respective localities Background subtraction techniques : Object detection can be achieved by building a representation of the scene called the background model and then finding deviations from the model for each incoming frame Segmentation ti : partition the image into perceptually similar il regions Tracking Point tracking : Objects detected in consecutive frames are represented by points Kernel tracking : Objects are tracked by computing the motion (parametric transformation such as translation, rotation, and affine) of the kernel in consecutive frames Silhouette tracking: Such methods use the information encoded inside the object region (appearance density and shape models)

7 Our Work Propose a highly parallel hardware implementation of an object tracking algorithem Improve the object region identification performance of the object tracking algorithm Hardware implementation achieves up to 100X speedup over the software execution

8 Outline Introduction Object Tracking Framework Searching Object Map Techniques Cascade Boundary Joiner Experiment Results

9 Main architecture Camera Input Memory (DDR2) + Monitor Main searching Tracking position output Camera input Memory (DDR2) + Monitor Preprocessing Preprocessing Main searching Tracking position output

10 FPGA-based object tracking system CCD CAM DDR2 Pre processing RAW to RGB format 2D Haar Transformation BGRND Current BRAM BRAM Classification Generator Main Searching Dilation & Erosion Search Map Search Path Splitter Path1 Path2... Path12 Single Square Single Square Single Square Path1 BRAM Path2 BRAM Path12 BRAM Five-level boundary joiner Tracking & Display Position Information Output

11 Outline Introduction Object Tracking Framework Searching Object Map Techniques Cascade Boundary Joiner Experiment Results

12 Searching Techniques Introduction Memory (0,0) (1,0) (0,1) (2,0) (1,1) (0,2) (2,1) (1,2) (3,3)

13 n n Index Index Searching Map Improvement Parallel Object Map Processing Split object map into 12 20x20 sub-matrices Dual- Mode Object Identification Single Mode Block Mode

14 Object Map Search Path Splitter Path1 Path2... Path12 Path1 BRAM Path2 BRAM Path12 BRAM Memory (0,0) 0) (1,0) (0,1) (1,1) 3.8 ms 2.66ms

15 Outline Introduction Object Tracking Framework Searching Object Map Techniques Cascade Boundary Joiner Experiment Results

16 Cascade Boundary Joiner Path 1 Path 2 Path 3 Path 12. Physical information recovery FORAM 5-stage boundary joiner

17 Cascade Boundary Joiner A B C A1 B1 A1 A2 B2 A3 B3 A2 A3 A1 A2 Complete object position

18 Outline Introduction Object Tracking Framework Searching Object Map Techniques Cascade Boundary Joiner Experiment Results

19 Experiment Setting Camera input Master Template A V A L O N DDR2 + Monitor Preprocessing Main searching Tracking position output

20 Experimental Results TABLE I. PERFORMANCE COMPARISON: SW VERSUS HW Object # SW Exec. Time SW fps HW Exec. Time HW. fps Speedup 0 30ms ms X 1 79 ms ms X ms ms X 3 392ms ms X 4 546ms ms X 5 857ms ms X ms ms X TABEL II. 20-FRAME PERFORMANCE COMPARISON Hardware Ex. Time Software Ex. Time Speedup ms 6038ms 62X

21 Conclusions and Future Work We presented an FPGA implementation for object tracking in color video a highly parallel hardware implementation object tracking through a dual search technique efficient boundary joining up to 100X speedup over the software execution Future work improve the sensitivity of the tracking algorithm to the luminance of the scene 3D image reconstruction ti

22 Thank you!