Copyright Warning & Restrictions

Transcription

1 Copyright Warning & Restrictions The copyright law of the United States (Title 17, United States Code) governs the making of photocopies or other reproductions of copyrighted material. Under certain conditions specified in the law, libraries and archives are authorized to furnish a photocopy or other reproduction. One of these specified conditions is that the photocopy or reproduction is not to be used for any purpose other than private study, scholarship, or research. If a, user makes a request for, or later uses, a photocopy or reproduction for purposes in excess of fair use that user may be liable for copyright infringement, This institution reserves the right to refuse to accept a copying order if, in its judgment, fulfillment of the order would involve violation of copyright law. Please Note: The author retains the copyright while the New Jersey Institute of Technology reserves the right to distribute this thesis or dissertation Printing note: If you do not wish to print this page, then select Pages from: first page # to: last page # on the print dialog screen

2 The Van Houten library has removed some of the personal information and all signatures from the approval page and biographical sketches of theses and dissertations in order to protect the identity of NJIT graduates and faculty.

3

4

5

6

7

8 BIOGRAPHICAL SKETCH Author: Degree: Sheetal Rajgure Doctor of Philosophy Date: December 2017 Undergraduate and Graduate Education: Doctor of Philosophy in Computer Science, New Jersey Institute of Technology, Newark, NJ, 2017 Master of Science in Computer Science, New Jersey Institute of Technology, Newark, NJ, 2009 Bachelor of Engineering in Instrumentation & Control University of Pune, India, 2002 Major: Computer Science Presentations and Publications: Sheetal Rajgure, Krithika Raghavan, Vincent Oria, Reza Curtmola, Edina Renfro- Michel, Pierre Gouton, Indexing multimedia learning materials in ultimate course search., Content Based Multimedia Indexing, 1-6, Sheetal Rajgure, Vincent Oria, Krithika Raghavan, Hardik Dasadia, Sai Shashank Devannagari, Reza Curtmola, James Geller, Pierre Gouton, Edina Renfro- Michel, Soon Ae Chun, UCS: Ultimate course search, Content Based Multimedia Indexing, 1-3, Sheetal Rajgure, Vincent Oria, Pierre Gouton, Slide localization in video sequence by using a rapid and suitable segmentation in marginal space., Color Imaging: Displaying, Processing, Hardcopy, and Applications, Duy-Dinh Le, Xiaomeng Wu, Shin ichi Satoh, Sheetal Rajgure, Jan C. van Gemert, National Institute of Informatics, Japan at TRECVID 2008, Text Retrieval Conference in Video 2008 iv

9

10

11

12

13

14

15

16 β β

17

18

19

20

21

22

23

24

25

26 19 th

27 19 th 20 th

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48 3 3

49

50

51 ( n ) (1/p) x p = x p i i=1 χ 2 (i, j) = M (I i (k) I j (k)) 2 k=1 I i (k) I i I j i th j th

52 Dist i T c, Dist i > T c,

53

54 m 1 = 1 N i Hist(i) m k = 1 N i (Hist(i) m 1 ) k i th ECR = ECR(n, k) = max( Xin, Xout nk ). σ n σ nk Xin σ n σ n 1 n n 1

55

56 L(x, y, σ) I(x, y) L(x, y, σ) = G(x, y, σ) I(x, y) σ G(x, y, σ) G(x, y, σ) = 1 (x2 + y 2 ) (2πσ 2 ) e( ) 2σ 2 D(x, y, σ) = (G(x, y, kσ) G(x, y, σ)) I(x, y) = L(x, y, kσ) L(x, y, σ) L(x, y, kσ) G(x, y, kσ) kσ σ

57

58 dist = 1 (Numberofmatchedkeypoints) (T otalnumberof keypoints)

59 E f = (C 1, C C M ) d f = E f E f+1 D LL = M d f (k) k=1

60 C... C E E

61

62 T = µ + t f ( σ) µ t f σ t f

63

64

65 cos((2k 1)(m 1)π, 6

66

67 M = α(i S )I H + (1 α(i S ))I v α(i S ) = 1/π[tan (β(i S S 0 ))] S (0 S 255) β(0.05 β 0.5) S β β

68 β β

69 Table 4.1 Mean Color Image Obtained after Clustering in DCT, Marginal, and Grayscale Space Original Image Ground Truth DCT Marginal Grayscale (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) thresholds. For DCT, K-means clustering is performed on dimension 1 and dimension 2 separately and then both the regions are merged. We use Berkeley dataset [63] and ground truth for general comparison. For some images we compute the ground truth ourselves. The visual comparison results of K-means are formulated in Table A Similarity Measures To compare different segmentation results from DCT, marginal and grayscale, we make use of similarity measures. First, we compute a confusion matrix between the ground truth and the regions obtained in segmentation and then we compute the following measures. 52

70 J = A B A B (β) = (1 + β2 )(P recision Recall) β 2 (P recision + Recall) β β S 0

71 S 0 S 0 S 0

72 Table 4.2 Mean Color Image Obtained after Clustering in Marginal and Grayscale Space Original Image Marginal Grayscale (i) (ii) (iii) (iv) (v) (vi) Heuristics for Slide localization Two heuristics are used to detect the slide regions from the regions obtained by k-means performed on marginal space. The heuristics are proposed, according to the standard observations in a recording of lecture video (Table 4.2). Size: In a lecture recording, slide usually covers the significant part of the video frame. In practice, the slide covers at least (1/4) of dimensions of a video frame. Luminance: In any presentation, the slide region is the most illuminant region than the surrounding. Based on the results obtained in Marginal space in Table 4.2, the first heuristic is used to calculate the size of each region. An adaptive threshold is set, based on the 55

73

74

75 Table 4.3 Saliency Vs Localization on Video frame for Slide extraction Original Image Slide extracted with Saliency (i) (ii) (iii) (iv) (v) (vi) 58 Slide extracted with Localization

76

77

78

79

80 Score(q, d) = t q tf(t in d) idf(t) weight title + t q tf(t in d) idf(t) weight body log N weight df title > weight body weight title Score(q, d) Score(q, d) weight presentation

81

82

83

84

85

86

87

88

89

90

91

92 C 1

93 C 2 C 1 C 2 G C = (V C, E C ) V C E C V i V j V i V j

94 G D = (V D, E D ) V D E D V i V j VD i V j D V D i V j D V D i V j D

95 C u = C 1, C 2... C n K u = Vc 1,... Vc n C u K u V 1 c C 1

96 {chapter{1, 2}, {16, 17}, {20, 21, 22}, {23, 24, 25}} {chapter{1, 2}, {5}, {6, 7}, {23, 24, 25}} {chapter{1, 2}, {5}, {6, 7}, {20, 21, 22}, {23, 24, 25}}

97

98 q =< t 1, t 2...t m > Score(q, ch) = t q tf(t in ch) idf(t) weight chaptertitle + t q tf(t in ch) idf(t) weight sectiontitle + t q tf(t in ch) idf(t) weight body weight chaptertitle weight sectiontitle weight body t f (t ch) log N d f Ψ(q) = {C t i }

99 Ψ(q) S C ϕ ϕ(g C, C) = {G C : G C G C } ϕ(g i C, Ci ) G C = (V C, E C ) : V C V C, E C E C L q L q = {G i C,...G k C } k th G 1 C...Gk C L q K u K u K u = {G i C, G j C...Gn C} G i C

100 λ λ(l q, K u ) : L q K u V q V u {L q \ K u } L q K u G C = {G 1 C, G2 C, G3 C, G4 C } G 2 C G 1 C, G2 C, G3 C

101 {G 1 C, G2 C } {G 1 C, G2 C } G q { }

102

103

104

105

106

107 (135069) JI=0.39, FM = 0.96 Time: JI=0.55, FM = 0.68 Time: S0median : 81 Ideal : JI=0.51, FM = 0.84 Time: (138078) JI=0.09, FM Time: 4.59 JI=0.09, FM = 0.86 Time: S0median 16 Ideal: 51 JI=0.06, FM = 0.56 Time: (161062) JI=0.23, FM = 0.97 Time: 4.22 JI=0.10 FM = 0.23 Time: S0median 30 Ideal: JI=0.12, FM = 0.96 Time: 3.70 (172032) JI: 0.06 FM: 0.71 Time: 4.87 JI: 0.14 FM: 0.89 Time: 2.71 Somedian : 29 Ideal:1 50 JI: 0.09 FM: 0.80 Time: 3.22 (187029) JI: 0.09 FM: 0.77 Time: 4.87 JI: 0.12 FM: 0.94 Time: 2.06 Somedian = 33 JI: 0.11 FM: 0.87 Time: 2.32 (189003) JI: 0.01 Time: 5.2 JI: 0.07, FM: 0.69 Time: 3.16 S0median 27, Ideal: 1-50 JI : 0.08, FM : 0.58 Time: 3.8 (24004) JI=0.05, FM = 0.42 Time: 5.7 JI=0.05, FM = 0.47 Time: 3.73 S0median 37 JI=0.037, FM = 0.43 Time:

108 (299091) JI: 0.11 FM: 0.65 Time: 3.52 JI: 0.13 FM: 0.79 Time: 1.81 Somedian : 43 Ideal: 1100 JI: 0.13 FM: 0.79 Time: 1.81 Somedian : 43 Ideal: 1100 (302003) JI: 0.22 FM: 0.94 Time: 3.6 JI: 0.26 FM: 0.97 Time: 2.48 Somedian : 46 Ideal:101 JI: 0.34 FM: 0.81 Time: 2.33 (323016) JI: 0.06 FM: 0.49 Time: 3.8 JI: 0.13 FM: 0.68 Time: 3.26 Somedian :26 Ideal: 1 50 JI: 0.09 FM:0.62 Time: 2.5 (368078) JI=0.08, FM = 0.71 Time: 5.7 JI=0.08, FM = 0.64 Time: S0median 60, Ideal: JI=0.10 FM = 0.76 Time: 3.86 (372047) JI=0.03, FM = 0.42 Time: JI=0.04 FM = Time: S0median : 44 Ideal: 1-50 JI=0.05, FM = 0.54 Time: (372047) JI=0.22, FM = 0.94 Time: JI=0.19, FM = 0.69 Time: S0median 32 Ideal: 101 JI= 0.22, FM = 0.88, Time:

109

110

111

112

113

114

115

116