Image information measurement for video retrieval

Transcription

1 37 2 Vol.37 No Journal on Communications Feruary 2016 doi: /j.issn x ,2 3,4 3,4 3,4 1,2 ( SEII map 4.4% 1.5 TP37 A Image information measurement for video retrieval YUAN Qing-sheng 1,2, ZHANG Dong-ming 3,4, JIN Guo-qing 3,4, LIU Fei 3,4, BAO Xiu-guo 1,2 (1. Institute of Information Engineering, Chinese Academy of Sciences, Beijing ,China; 2. National Computer Network Emergency Response Technical Team/Coordination Center of China, Beijing , China; 3. Key La of Intelligent Information Processing, Chinese Academy of Sciences, Beijing , China; 4. Institute of Computing Technology, Chinese Academy of Sciences, Beijing ,China Astract: To meet the speed and performance requirements, Su-region entropy ased image information measurement (SEII) method was proposed, which integrates the salie region detection, region division and features fusion. And, performance evaluation method was designed and many experiments were carried out, proving SEII coordinates with human vision evaluation. Also, SEII is evaluated in a real video retrieval system, which shows increase aout 4.4% of map with 1.5 times speedup. Key words: video retrieval, key frame selection, image information, salient region, features fusion [1] [2] [3] 2 [4] (a)~ 1(c) 1(d)~ 1(f) 1(a)~ 1(c) 1(d)~ 1(f) dmzhang@ict.ac.cn No , No , No No.2013AA Foundation Items: The National Natural Science Foundation of China (No , No , No ), The National High Technology Research and Development Program of China (863 Program)(No.2013AA013205)

2 2 81 (a) 1 () 2 (c) 3 (d) 4 (e) 5 (f) 6 1 1(d)~ 1(f) 3 1(a)~ 1 (c) D P= 2 D [0, P- 1] (1) SEII, su-region entropy ased image information measurement 2 1) 2) H ( f) i j p i,j ( i, j) P 1 P 1 H p log( p ) (1) = i 0 j = 0 i, j i, j 2.1 Koch Ullman [6] Itti 1 2 [5]

3 82 37 Ma [8] Zhai Cheng [10] [11] 2 Q = { q1, L, qb } h ( ) = { h } Dp D [1, B] p, h D q q Q B =64 [12] (2) Q D k k k k k [7] p D p p [9] C( D ) = 1 p [6] h ( ) h ( ) h ( ) h ( ) I ( i, j) I ˆ( i, j) ( i, j) (2) I ( i, j) m( i, j) Iˆ( i, j ) = (3) s ( i, j) + C i {1, 2, L, M } j (1, 2, L, N ) M N C= 1 m( i, j) s ( i, j) K m( i, j) = w I ( i + k, j + l) s k, l ( i, j) = K L 2 w [ I ( i + k, j + l) m( i, j)] k. l (4) k = K l = L w = { w k = K, L, K, l = L, L, L} k, l D p D q D q Q D p D q L k = K l = L K=L= (a) () (c)

4 [20] 2 (5) w c w t D p k k C ( D p ) T ( D p ) T( D p ) 3 [13,14] [15,16] [17] [18] 7 3 3(c) [19] Wc Wc F[ C( D ), T ( D )] = C( D ) + T ( D ) p p p p W + W W + W D p c t c t (5) X c d (support vector regression) SVR SVR X a X µ X l [21] X s (a) () (c)

5 for each image of M N pixels do 3 for each region D do for each feature F { r, g, T} do end for end for end for sl=vector of the sum of F row of region D hl=histogram of sl on =entropy( hl) hc=histogram of sc =entropy( hc) =entropy( h) =mean of 4 5 SVR 3 R r = R + G + B G g = R + G + B T = R + G + B X l on value for each pixel of each ins value for each pixel of ins X s =std of F d =proportion of saliency pixels N sc=vector of the sum of F each column of region D X c h=histogram of F X a X µ F M M N 3 ins ~ ~ ~ ~39 5 0~ % [23] 20% 10 SROCC [24] Peng [13] [13] [22]

6 2 85 (a)33.6 () 49.7 (c) 53.6 (d)62.3 (e)87.5 (f)5.0 6 SROCC 2 [13] SROCC / / / / / / / / / / / / MAP, mean average precision CPU Intel Core i5 3.1 GHz 4 GB Windows7 TRECVID A G B H C I D VOA J E K F Student CNN 7 aseline 15 map 4.4% A D E F G H J

7 map 8 I [1] SHAHRARAY B, GIBBON D C. Automatic generation of pictorial transcripts of video programs[c]//proc SPIE. c1995: [2] WOLF W. Key frame selection y motion analysis[c]//acoustics, Speech, and Signal Processing, International Conference, c1996: [3]. [M]. :, ZANG Y J, Content ased video information retrieval[m]. Beijing: Science Press, 2003 [4] SAAD M A, BOVIK A C, CHARRIER C. Blind image quality assessment: a natural scene statistics approach in the DCT domain[j]. IEEE Transactions on Image Processing, 2012, 21(8): [5] KOCH C, ULLMAN S. Shifts in selective visual attention: towards the underlying neural circuitry[c]//matters of Intelligence. c1987: [6] ITTI L, KOCH C, NIEBUR E. A model of saliency-ased visual attention for rapid scene analysis[j]. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1998,20(11): [7] MA Y F, ZHANG H J. Contrast-ased image attention analysis y using fuzzy growing[c]//the eleventh ACM international conference on Multimedia. c2003: [8] ZHAI Y, SHAH M. Visual attention detection in video sequences using spatiotemporal cues[c]//14th annual ACM international conference on Multimedia. c2006: [9] CHENG M M, ZHANG G X, MITRA N J, et al. Gloal contrast ased salient region detection[c]//computer Vision and Pattern Recognition(CVPR). c2011: [10] HOU X, HAREL J, KOCH C. Image signature: highlighting sparse salient regions[j]. IEEE Transactions on, Pattern Analysis and Ma

8 2 87 chine Intelligence, 2012, 34(1): [11] MAI L, NIU Y, LIU F. Saliency aggregation: a data-driven approach[c]//ieee Conference. Computer Vision and Pattern Recognition(CVPR). c2013: [12] BHATTACHARYYA B A. On a measure of divergence etween two statistical populations defined y proaility distriutions[j]. Bulletin of the Calcutta Mathematical Society, 1943, 35:99-110, [13] PENG J, QING X L. Keyframe-ased video summary using visual attention clues[j]. IEEE MultiMedia, 2010, 17(2): [14] LAI J L, YI Y. Key frame extraction ased on visual attention model[j]. Journal of Visual Communication and Image Representation, 2012, 23(1): [15] HUA X S, ZHANG H J. An attention-ased decision fusion scheme for multimedia information retrieval[c]//advances in Multimedia Information Processing-PCM Springer Berlin Heidelerg, c2005: [16] MA Y F, HUA X S, LU L, et al. A generic framework of user attention model and its application in video summarization [J]. EE Transactions on Multimedia, 2005, 7(5): [17] HU Y, XIE X, MA W Y, et al. Salient region detection using weighted feature maps ased on the human visual attention model[c]//advances in Multimedia Information Processing-PCM Springer Berlin Heidelerg, c2005: [18] ARMANFARD Z, BAHMANI H, NASRABADI A M. A novel feature fusion technique in saliency-ased visual attention[c]//advances in Computational Tools for Engineering Applications, c2009: [19] LAI J L, YI Y. Key frame extraction ased on visual attention model[j]. Journal of Visual Communication and Image Representation, 2012, 23(1): [20] EJAZ N, MEHMOOD I, WOOK B S. Efficient visual attention ased framework for extracting key frames from videos [J]. Signal Processing: Image Communication, 2013, 28(1): [21] SMOLA A J, SCHÖLKOPF B. A tutorial on support vector regression[j]. Statistics and Computing, 2004, 14(3): [22] PARK J S, CHEN M S, YU P S. Using a hash-ased method with transaction trimming for mining association rules[j]. IEEE Transactions on Knowledge and Data Engineering, 1997, 9(5): [23] MITTAL A, MOORTHY A K, BOVIK A C. No-reference image quality assessment in the spatial domain[j]. IEEE Transact ns on Image Processing, 2012, 21(12): [24] SHEIKH H R, BOVIK A C. Image information and visual quality[j]. IEEE Transactions on Image Processing, 2006, 15(2):