EE 6882 Statistical Methods for Video Indexing and Analysis

Size: px

Start display at page:

Download "EE 6882 Statistical Methods for Video Indexing and Analysis"

Maria Chambers
5 years ago
Views:

1 EE 6882 Statistical Mthods for Vido Indxing and Analysis Fall 2004 Prof. Shih-Fu Chang Lctur 3 Part B (9/5/04) Exapl of E-M: Machin Translation Brown t al 993 A translation odl btwn English and Frnch And th 2 progra 3 has 4 bn 5 iplntd 6 L progra 2 a 3 t 4 is n application 5 Assu ach Frnch word is translatd into on or zro English word. Th abov assuption ay not b valid. But us it for now. Othr alignnt odls xist. Can w ust count co-occurrncs? Probl: Givn a Frnch passag (string) f want to find th bst translation in English * = argax Pr( f) = argax Pr( f ) Pr( ) 2 translation odl prior languag odl Nd to stiat Pr( f ) 'Translation Modl'

2 Probabilistic Gnration Modl Alignnt: Hiddn Variabl English string = 2 l l words Frnch string f = ff2 f words i And th 2 progra 3 has 4 bn 5 iplntd 6 Alignnt a = aa a = i L progra 2 a 3 t 4 is n application f 2 a a = i : -th word in Frnch string is 5 connctd to i-th word in f i Gnration Modl: Pr( f ) = Pr( f, a ) a = pick th connctions connctions English string a = 0 : not connctd to any word a spcifis th connctions btwn words in and f Pr( f, a ) = Pr( ) Pr( a a, f,, ) Pr( f a, f,, ) pick th word 3 A vry sipl odl Pr( ) = ε constant Pr( a a, f,, ) = l + Pr( f a, f,, ) = t( f ) a Diffrnt word lngths qually likly Evry position is qually likly Dos not considr word ordrs Only considr th (Frnch English) word pairs ε Pr( f, a ) = t( f a ) ( l + ) = l l ε ε Pr( f ) = t( f ) t( f ) = ( l+ ) ( + ) a a 0 0 a = l a = a = = ( (l) trs) Find opti al t ( f ) to axi iz Pr( f ) subct to constraint t ( f ) = f 4 2

3 Constraind Optiization- Lagrang Multiplirs ht (, ) Pr( f ) ( t ( f δ λ = λ ) ) St (, ) 0 t( f ) ht λ = δ f t f a f f f ( ) = λ Pr(, ) δ(, ) δ(, a ) a = prob. of alignnt a(hiddn) # of tis conncts to f in alignnt a = λ c( f ; f, ) M-Stp xpctd # of tis conncts to f in translation ( f ) (Too any trs, (l) ) Pr( f, a ) Pr( a f, ) = = Pr( f ) = t( f ) t( f ) a a = 5 a E-Stp Rducing th coplxity l l l ε ε ε Pr( f ) = t( f ) = t( f ) = t( f ) ( l+ ) ( l+ ) ( l+ ) a a 0 0 i a = a = a = = = i= 0 xapl : t t t + t t t + + t t t (8 trs) = ( t + t )( t + t )( t + t ) Us th sa procss of constraind optiization Pr( f, a ) Pr( a f, ) = = Pr( f ) = l = i= 0 t( f ) t( f ) l t( f ) c( f ; f, ) = δ( f, f ) δ(, ) t( f 0) + t( f ) + + t( f l ) t( f ) = λ c( f ; f, ) xpctd # of tis conncts to f in translation ( f ) 6 a i E-Stp i = i= 0 count of f in f count of in M-Stp Much or fficint, only O(l) trs 3

If thr ar ultipl translation sapl pairs (S) Thr will b log_of_su in th prob. quation. Us E-M to prfor optiization. Tak drivativ of aux. function instad of liklihood.

4 If thr ar ultipl translation sapl pairs (S) Thr will b log_of_su in th prob. quation. Us E-M to prfor optiization. Tak drivativ of aux. function instad of liklihood. Siilar solution S c( f ; f, ) = c( f ; f, ) : S : S s s S s = Not: thr ar othr odls for MT that considr ordrs of words. (s Brown 93) 7 Obct Rcognition as Machin Translation Duygulu t al ECCV 2002 Annotatd Iag St + Iag Rgion sgntation Forulation: Translation btwn iag rgions and annotation words Givn rgions in an iag, prdict th likly words Assu ach word is connctd to on rgion. But th actual connction is hiddn can stiat th soft probability. Iag rgions ar clustrd to for discrt tokns (visual vocabulary). 8 4

5 Notations 9 Forulation Liklihood Modl Paratrs θ N M L n n p({ w} { b}) = p( a = i) t( w= w b= b ) n= = i= n n ni Su ovr all possibl alignnts p( a = i) : prob. of conncting word to rgion i in iag n n twb ( ) : prob. of gnrating w givn rgion b Not: this corrspond to Modl 2 of Brown. p(a n =i) dpnds on positions of words and rgions Hiddn Variabl an = { an, an2,, anm n } a = i if -th word conncts to rgion i in iag n n In Modl : Pr( a a, f,, ) = l + 0 5

6 E-M for Paratr Estiation * find optial θ = argax log pwb (, θ) θ log liklihood N Mn Ln log pwb (, θ ) = log( pa ( = itw ) ( = w b= b )) n= = i= Hard to tak drivativ, instad us E-M n n ni Auxiliary Function Q ( old ) ( θθ ; ) = Us currnt odl to stiat hiddn var. N Mn Ln ( old ) p an = i wn bni θ p an = i t w= wn b= bni n= = i= Su ovr th tru oint prob. (,, )log[ ( ) ( )] Constraind optiization Nd to satisfy constraints: * w i tw ( b) = b * * * pa ( = i) = word in any iag n n not p( a = i) ar th sa for all iags of th sa siz ( L and M ) n n n Lagrang Multiplirs L= Q( θθ ; ) + α ( p( a = i)) + β ( t( w b )) ( old ) * * nl,, n * b α * * nl,, i b w Tak th drivativs to paratrs, θ 2 6

7 Exprints 4500 Corl iags, 4-5 words pr iag, 37 words in total Rgions obtaind using Noralizd Cuts thod (Shi & Malik 97), 5-0 rgions pr iag, siz thrsholdd, clustrd to for 500 blobs k-ans 33 rgion faturs Rgion color and standard dviation Avg. orintation nrgy (2 filtrs) Rgion siz, location, convxity, first ont Ratio of ara to (boundary lngth) 2 4 7

8 Tsts Auto Annotation 500 iags hld out as tst st for ach rgion, coput ost likly word = * w argax ( ni ) w if p( w b ) > Th ni p w b thn iag n is annotatd with word w If Th=0, thn vry blob prdicts a word Rsults Only 80 words fro th 37 word vocabulary ar prdictd Othrs do not gt th ax prob. for any blob Evaluation tric: prcision rcall of iag rtrival Using th autoatically annotatd words Th actual annotations ar usd as ground truth Not this is not rgion lvl annotation 5 * Words with good prdiction 6 8

9 Issus Copound words not addrssd E.g., polar bar So words ar clustrd and should not b distinguishd E.g., (kit, hors, ar, foal), (laf, flowr, plant, vgtabl) So visual blobs ar confusing. Faturs ar liitd E.g., agl obcts and airplan obcts, grass and undr watr plants Txt and visual faturs ar sparatly procssd Joint clustring ay hlp E.g., Hirarchical ixtur odl (Hoffan) 7 Rgion lvl annotation Evaluat th accuracy of txt-rgion corrspondnc * w = argax p( w b ni ) w Mtric: % of th ti corrct words ar prdictd for a rgion Good prdictions: sky, tr, grass High rcall, i.., consistnt visual faturs and co-occurrnc with txt Prob. for othr words ar low for th sa visual blob May rfus to prdict words (i.., null word) by incrasing th prob. thrshold Word clustring ay hlp iprov th accuracy 8 9

10 Exapls of word prdiction Satisfactory Unsatisfactory 9 Effct of null word 20 0

11 Effct of word clustring 2

2008 AP Calculus BC Multiple Choice Exam

2008 AP Calculus BC Multiple Choice Exam 008 AP Multipl Choic Eam Nam 008 AP Calculus BC Multipl Choic Eam Sction No Calculator Activ AP Calculus 008 BC Multipl Choic. At tim t 0, a particl moving in th -plan is th acclration vctor of th particl