Automatic Segmentation, Aggregation and Indexing of Multimodal News Information from Television and the Internet

Transcription

1 Automtc Segmentton, Aggregton nd Indexng of Mutmod News Informton from Teevson nd the Internet Murzo Montgnuoo, Aberto Messn,, Roberto Borgoto RAI Rdoteevsone Itn Centre for Reserch nd Technoogc Innovton C.so Gmone 68, I-035 Torno (Ity) {murzo.montgnuoo,.messn, Journ of Dgt Informton Mngement Unverstà deg Stud d Torno Deprtment of Computer Scence C.so Svzzer 85, I-049 Torno (Ity) messn@d.unto.t ABSTRACT: The gob dffuson of the Internet hs enbed the dstrbuton of nformtve content through dynmc med such s RSS feeds nd vdeo bogs. At the sme tme, the decresng cost of eectronc devces hs ncresed the pervsve vbty of the sme nformtve content n the form of dgt udovsu dt. Ths rtce presents system for the rge-sce unsupervsed cquston, segmentton nd ndexng of TV newscsts. In prtcur, t dscusses the prncpes nd performnce of the prts of the system dedcted to the detecton nd segmentton of progrmmes from the cqured strem. In ddton to the core technoogy, we so ntroduce nd dscuss nove method for ssessng the resuts of story boundres segmentton gorthms, bsed on user-vdted mesurement. Due to the heterogenety of current news dstrbuton chnnes, further nnovtve spect of ths rtce s the descrpton of frmework for mutmod nformton ggregton. The core of ths frmework s cross-mod custerng process for whch nove, symmetrc smrty mesure s provded. The mpemented prototype uses onne news rtces nd TV news progrmmes s nformton sources, nd provdes mutmod servce ntegrtng both contrbutons. Experment evuton of the system proves the effectveness of the method n the studed cse. Ctegores nd Subject Descrptors H. 4.0[Informton Systems Appctons]: Gener; H 3.[Informton Storge nd Retrev] Content Anyss nd Indexng; I 5.3[Pttern Recognton] Custerng Gener Terms: Web Indexng, Internet, TV News Indexng Keywords: Content segmentton, Mutmod ggregton, Mutmed nnotton, Semntc enrchment Receved: My 00: Revsed 7 June 00; Accepted 7 June 00. Introducton nd Reted Work Modern nformton socety s chrctersed by rpd evouton of nformton consumpton modes, nd by n overwhemng mount of mutmed dteng produced every dy, nd devered through sever mutmod nformton chnnes, mong whch dgt nterctve teevson s st mjor one. It s now commonpce tht nterctve teevson, n ts dfferent nd vregted nterprettons nd embodments (e.g., IPTV, Web TV) s seen s the next fronter of med producton busnesses. However, expotng the fu potent of puttng users n the oop s st jeoprdsed by the mted cpbty of trdtonrodcsters of turnng ther estbshed workf ows nto somethng dpted to the new scope. In such hghy dynmc context, the convergence of Internet technoogy nd dgt teevson represents prncp drver towrds the utomtston of the producton processes to fuf the requrements of the new emergng ppcton domns. As f rst step n such utomtston process, utomtc progrmme segmentton concerns the bty of utomtcy detect semntcy coherent prts of teevson progrmmes. Ths step s key enber for nterctve ppctons tht ncude nformtve content mngement, snce the bty of correcty nd eff centy segmentng news content s the bss on whch sever other ppcton cn be deveoped, e.g. recommendton systems, users preferences coecton prof ng, personsed home TV ppctons. Some reference works n ths re re those presented n [8, 0, 5]. In prtcur, to sove the news story segmentton, the common bse of the pproches s consttuted by the use of combnton of vsu, udo nd speech fetures. The TRECVID nttve hd news segmentton mong ts tsks n 003 nd 004. The works n [7, ] present the vrous pproches dentf ed nd deveoped by the TRECVID prtcpnts n those two seres. These pproches ncuded ether vdeo nd udo chnnes nyss or, n ddton, speech-to-text utomtc trnscrpts. The bsene fetures empoyed n sever cses re vsu smrty between shots wthn tme wndow nd the tempor dstnce between shots, e.g. [9]. Other heurstcs ke smrty of fces pperng n the shots nd the detecton of the repeted ppernce of nchorperson shots cn dd suppement yer of nformton to mprove the over ccurcy [5, 9, ]. The udo chnne contrbuton cn be empoyed to detect puses, potentoundres for topc chnges [9, 6, 9], or to detect chnges n udo cssf cton ptterns (e.g., musc to speech chnges [9, 6]), or to detect speker chnges [6]. As thrd nformton source, text from trnscrpts or utomted speech recognton s very often used, ether by serchng smr word ppernces n dfferent shots or by detectng text smrtes between the shots [5, 9]. A f rst crtc spect connected wth these technooges s reted to ther effectve nd eff cent mpementton n re-fe domns, where opertng condtons nd dt chrcterstcs my sgnf cnty vry from prototype envronments. Another crtc spect s specf cy connected wth the segmentton gorthm,.e. the rgorous nd robust evuton of ts ccurcy. Evuton systems bsed on exct boundry mtches my be Journ of Dgt Informton Mngement Voume 8 Number 6 December

2 tte rebe on expressng re effectveness of segmentton gorthm. In fct, they do not consder ny ntermedte reevnce, nd news tem s def ned correcty retreved ony f t s excty mtchng the reference vues, even f there s ony few seconds tme g between detected nd true segments. The TRECVID evuton strtegy extends ths pproch by ddng f xed uncertnty threshod of 5 seconds to ech boundry. However, ths symmetrc wndow ck n tkng nto ccount the dfferent senstvty of users to strtng nd endng story boundres, s we s the dfferent objectve weght tht hs to be ssgned to extr or mssng mter. A second crtc spect regrds the methodooges used to ggregte nd present the rge mounts of nformton vbe from dfferent sources nd heterogeneous med formts. In fct, the gob dffuson of the Internet nd the evouton of Web technooges re enbng the creton nd deverng of dynmc nformtve contents such s news, bogs or vodcsts. Those contents re usuy dstrbuted through RSS feeds, n XML-bsed formt orgny proposed by Netscpe to provde smpe, extensbe nd f exbe content dstrbuton []. Users cn mnge RSS feeds usng feed reder tht perodcy downods the updted contents from the subscrbed feeds, dspys the tems n ech feed nd provdes nks to the reted resources. However, returnng the unorgnsed st of tems ncuded n the subscrbed feeds ends n cusng n nformton overod effect. One souton to ddress ths probem conssts n ggregtng smr tems shrng some common ttrbutes. Poneer works used sts of keywords to descrbe the feeds contents []. More recent pproches use custerng technques nd knowedge ntegrton [6, 3]. The common bckground of such pproches s ther cpbty of performng utomted ggregton of dt coected from snge, cosed domn,.e. the Internet. So fr, however, ony few ttempts hve been mde to nvestgte the possbty of mergng nformton from heterogeneous nformton chnnes, such s teevson brodcsts nd Web pges. As n exmpe, the work n [] uses both ntern udovsu fetures nd dfferent extern nformton sources for event detecton n tem sports vdeo. Smry, the sme tsk s performed n [0, 3], by mkng use of web-cstng text nd brodcst vdeo nformton. Further exporng ths drecton, ths rtce descrbes n unsupervsed frmework for content-bsed TV news stores nd onne newspper rtces ggregton nd retrev. The rest of the rtce s orgnsed s foows. Secton descrbes the rchtecture of our prototype system. Secton 3 presents experment evutons. Fn concusons nd future work re ustrted n Secton 4.. Prototype System Archtecture Fgure ustrtes the hgh-eve rchtecture of our prototype system. The system cn be vewed s processng mchne wth two nput chnnes,.e. dgtsed brodcst news strems (DTV) nd Web RSS feeds (RSSF), nd one output chnne,.e. the mutmod ggregton servce (MMAS), whch s utomtcy determned from the semntc ggregton of the nput strems.. DTV Processng Chn The DTV processng sub-system ws desgned nd mpemented to offer n eff cent, scbe nd robust softwre rchtecture for the cquston nd utomtc nnotton of brodcst news, ncudng utomtc news story segmentton nd semntc nyss of spoken text wth Nmed Enttes Recognton nd Cssf cton (NERC). Frsty, the brodcst news strems re utomtcy detected nd segmented nto ther eementry Fgure. Hgh-eve prototype system rchtecture news stores. The detecton s performed usng the utomtc vdeo cp detecton technque descrbed n Secton... The segmentton process s done by expotng ur nd vsu cues, wth the hep of three-yered heurstc frmework, s presented n Secton... Once segmented, the udo trck of ech story s trnscrbed nd ndexed n the TV ctogue (see Secton..3), whch serves s the permnent repostory from whch fu text serch nd ctegory serch functontes re devered to the f n users... Detecton of progrmmes from ve strems To cheve utomtc segmentton of ve strems nto progrmmes we mke use of n optmsed vdeo cp mtchng technque. Vdeo eements (shots) ndctng strtng nd endng of progrmmes re used s reference prototypes to be serched through the cqured vdeo strems. The technque conssts of ernng phse foowed by detecton phse. In the ernng phse, for ech mcro-event of nterest E, e.g. progrmme strtng or endng jnge, N E nstnces {e, e,..., e NE } re seected from dy teevson progrmme cqustons. On ech nstnce e m n dptve threshod shot detecton gorthm bsed on dspced frme umnnce dfference (L-DFD) s performed. Adptvty s bsed on the oc sttstcs of L-DFD, so tht content hvng hgher L- DFD vrnce s processed gnst hgher threshods. After the shot detecton process, ech cp nstnce e m s spt nto N shots formng the set S s m {S, S,..., S N }, whose ength s rry s Sm (,,..., ), where N s n, n,..., N s s the number of frmes wthn the shot S n. Let F (f, f,..., f NF ) be the set of oweve vsu fetures extrcted from ech shot, whch ncudes hue, sturton, nd vue coour descrptors, umnnce, contrst nd drectonty texture descrptors [8], tempor ctvty moton descrptor. Ech eement of F s represented usng unformy dstrbuted B-bn hstogrm, where the st bn s used to count the pxes of the frme for whch the mesurement of feture returns n undetermned vue (e.g., hue for grey pxes). Therefore, ech mcro-event E s represented by the set A E (,,..., NE ), whch, n turn, s mde of N E rrys of feture vectors of sze Ns, Ech eement mn, n Ns of ech rry m A E s re mtrx of dmensonty N f B n, where we rec tht N f denotes the tot number of extrcted ow-eve fetures (.e., N f 7), B denotes the tot number of bns used n ech feture hstogrm (.e., B 65), nd n denotes the ength (n frmes) of the shot S n. 388 Journ of Dgt Informton Mngement Voume 8 Number 6 December 00

3 To mprove the eff cency of the detecton, seecton of the most promsng shots mong the Ns shots pertnng to the mcro-event E s performed. The seecton s bsed on the mesurement of the Bhttchry dstnce between the dstrbuton of the feture vectors extrcted from ech of the Ns shots nd reference set of shots R {r, r,..., r NR } tht represents the generc poputon of shots mong whch the mcro event E hs to be serched. Durng the shot seecton, feture seecton s so performed, n order to pck up the fetures mxmsng the mesured dvergence. More formy, we seect subset of the set of detected shots S nd subset of the set of extrcted fetures F, bsed on the vue ssumed by the foowng functon: σ B jk jkm σ + σ μ σ B B jkm rkm jkm n + m σ 4 m jkm + jkmσ rkm σ j j h σ jkm j N E j h N E μ kmh j rkm rkm () μ () kmh j j NE NE kmh j j h N E N E kmh j Under the ssumpton tht the eements of the N f extrcted feture hstogrms foow ndependent Gussn dstrbutons, the eements Equton () s n pproxmton of the Bhttchry dstnce [6] between the dstrbutons of the fetures n the set of nstnces of the shots of the mcro-event E nd the dstrbutons of the fetures n the gener poputon of shots. Indces j;m vry n B, whe ndces k;h vry n N f nd n, respectvey. The vue of the coeff cent B jk (see Equton ) cn be tken s mesurement of the dstnctveness of feture f k F for the shot s j S m. Foowng ths ssumpton we cn therefore seect the top-k shots mong the vbe Ns shots w.r.t. ech of the N f vbe fetures. Ths seecton cn be represented rrngng the eements B jk n mtrx B nd seectng the coupes (s, f bj ), s S, f m b j F, correspondng to the eements of B rechng the top-k vues. Ths pproch enhnces the precson of the detecton phse, snce t seects the combntons shot/feture tht mxmse dvergence of the sttstc dstrbutons of reference shots w.r.t. generc poputon. However, ths s not yet enough to ensure the optmston of rec, snce seected feture dstrbutons my hve sgnf cnt stndrd devtons round ther men vues. To mt ths drwbck, we seect the best combnton of fetures mong N f the tot possbe N f (3) combntons, n order to expot dversty effects of dfferent fetures. The combnton s chosen s the one mxmsng the hrmonc men of the dvergence mesurement gven by the ndvdu fetures correspondng to the B j coeff cents. In the detecton phse, verged feture vectors j n N E N n k E jk n re used s fxed references, wth respect to whch cqured shots re compred usng dstnce mesurement bsed on hstogrm ntersecton. Let I be n cqured shot nd s J S, J...N s reference shot for j the mcro-event E, nd { I,...N se } j nd { sj,...n se } ther respectve sets of verged feture vectors, where N se s the number of seected fetures for the reference shot s J. The dstnce D(I, s j )between nd s J s def ned s foows: D N se N, J J ( I s ) d ( I, s ) se d B j j ( I sj ) mn( I, s ) J j (4), (5) The cqured shot I s cssf ed s n nstnce of the reference shot s J f: D α I, σ j ( I sj ) J ασ (6) J k where I s the ength of the shot nd α [0,] s prmeter governng the rec of the detecton... Segmentton of TV Newscsts After hvng cqured nd detected progrmme boundres foowng the pproch descrbed n Secton.., newscst progrmmes re mported nto the system nd segmented nto ther ogc unts,.e. news stores, whch re the eementry tems stored n the MA ndex (see Fgure ). Segmentton of newscsts progrmmes nto news stores s done expotng ur nd vsu cues wth the hep of three-yered heurstc frmework. The used heurstcs re bsed on the observton of the stystc nguge of set of 80 TV newscsts, tken n controed perod of tme from dy schedues of the 7 mjor ntonrodcst chnnes. The bsc heurstc (H), so dopted n terture by e.g. [8], conssts n consderng boundres of shots contnng the nchormn s equvent to news stores boundres. In order to detect the nchormn shots we use second heurstc (H), consstng n observng tht the most frequent speker s the nchormn nd tht (s)he speks mny tmes durng the progrmme, nd for perods of tme dstrbuted ong the progrmme tmene. Ths observton ows seectng the speker who most key s the nchormn, provded tht speker custerng process bes the spekers present n the progrmme nd ssoctes them to tempor segments of the content. However, the ppcton of the frst two heurstcs s not yet enough to dscern stutons where the nchormn ntroduces sever bref stores n sequence, wthout nterruptons fed wth extern contrbutons. To overcome ths mtton we use the thrd heurstc (H3),.e. knowng tht n the gret mjorty of observed cses the ntroducton of new bref story s ccompned by cmer shot chnge (e.g., from cose up shot to wder one). Thus, to optmse the ccurcy of segmentton, we perform shot custerng process bsed on both udo segmentton [4] nd vdeo fetures [4]. Ths ows us to detect nd cssfy shot custers s pertnng to studo shots contnng the nchormn foowng the sme frequency/extenson heurstc used for detectng the cnddte speker (H). Ths doube custerng process enbes very smpe nd effectve gorthm tht seects vdeo nd udo custers on the bss of ther mutu coverge percentge. jk J Journ of Dgt Informton Mngement Voume 8 Number 6 December

4 More formy, et C A {,,..., NA } nd c V {v, v,..., v NV ) be the set of speker custers nd the set of vdeo shot custers for detected newscst progrmme, respectvey. Ech eement of C A (nd smry of C V ) s represented s {, j : ext j [ts j, te j ]}, beng ts j, te j respectvey the strtng nd endng tmes of the j-th eement of w.r.t. the progrmme tmene. Be ε( ) functon returnng the extenson of custer, defned s: ( ) mx ( te ) mn ( ts ) ε (7) j j We seect C A (nd smry v k C V ) s pvot custer f the foowng condton prmetersed by the threshod τ p hods: () τ p ε > j j (8) Be C A p C A nd C V p C V the subsets of custers contnng ony the pvot eements seected through Equtons (7) nd (8). We further seect mong eements of C V p, the top-m scorng eements v, v,..., v M w.r.t. the tempor coverge wth eements of C A p, def ned s: COV v v ( v ) ength( ek em), (9) v k m where ength( ) returns the durton of ts rgument s per p* the ntutve def nton. Let C V be the further pruned verson p of C A, fter the seecton of the top-m eements. Then, story boundres re dentfed s those n whch n udo or vdeo custer boundry occurs mong the eements of C AV C p A C Vp *, wth threshod τ o to vod over-segmentton. Nmey, eements e, v of members of C AV re t f rst ordered w.r.t., ther strtng tmes ts, formng vector (e v,, e v,,..., e v N )., v Then, n eement e k [ts k, te k ] s dscrded f ts k, ts k- < τ o. To sum up, the prmeters of the segmentton gorthm re: τ p,m nd τ o. Fgure ustrtes n exmpe. The nchormn shots nd b re detected ccordng to the heurstc H becuse both contn the sme speker A. As shot boundry s detected between the shots nd b, the f rst two stores re segmented ccordng to H3. The succeedng stores re then detected ccordng to H. Fgure. Iustrton of news story segmentton..3 News Stores Indexng Spoken content of the detected news stores s extrcted usng speech to text engne bsed on [4], cpbe of trnstng both Itn nd Engsh. Subject cssfcton of stores s done usng nve Byesn cssfcton mode trned on corpus mde up of tems of extrcted text nd nnotted ccordng to txonomy of 8 ctegores. The corpus counts 5,000 tems, 4/5 of whch were used for trnng nd the remnng /5 for test. Over subject cssfcton ccurcy s 0.8, whe progrmme eve ccurcy,.e. verge cssf cton ccurcy ccuted on tems beongng to the sme progrmme, s Nmed entty recognton from trnscrbed text s so performed, usng the technoogy descrbed n [5].. RSSF Processng Chn The RSSF strem s consttuted by the RSS feeds of sever mjor onne newsppers nd press gences. An RSS feeds n consttuted by set of tems, ech descrbed by tte (.e., the hedne of the correspondng onne rtce), descrpton (.e., ref summry of the content of the correspondng onne rtce), pubcton tmestmp (.e., the dte nd tme when the rtce ws f rst pubshed on the newspper Web ste) nd nk to the correspondng fu rtce. Addton metdt coud be so ncuded, such s st of tem s ctegores, or some comments, ccordng to the RSS specf ctons []. Ech RSS tem cn be expressed s tupe ( uud, pubdte, nk, feed, tte, phrses), where uud s the unvers dentf er of the tem, feed s the nme of the source feed where the tem ws found nd phrses s the set of sentences ncuded n the tem s descrpton. A such tupes re stored s records n PostgreSQL dtbse. On ech RSS tem ngustc nyss bsed on POS tggng [7] s performed to dentfy the ngustc enttes, e.g. verbs, nouns, djectves, ncuded n the tte nd the descrpton phrses of the RSS tem. The outputs of the ngustc nyss re empoyed by the query constructor to generte set of representtve query expressons, s deted n Secton... The generted queres re then submtted to the ndex structure of the TV documents ctogue. For ech tem, the resut of ths serch operton s weghted set of newscsts stores of decresng ff nty to the trget query. On the resuts of such queres, cross-mod custerng process s performed to ggregte tems bsed on ther semntc smrtes, thus producng wht we c the mutmed dosser,.e. mutmod ggregton of TV newscst stores nd newsppers rtces shrng the sme semntc content. Further nytc dets on the custerng process re provded n Secton... For ech mutmed dosser unque dentf er s provded n ddton to the trnscrptons of the ncuded news stores nd the tte nd descrptons of the ncuded RSS tems. Ths nformton s f ny stored n the mutmod ggregton ndex (MA), whch s bsed on Lucene [5] nd t serves s the permnent dtbse from whch the mutmod ggregton servce s mntned nd devered to the f n users. The system supports fu-text queres nd even fed specfc queres (e.g. pubcton/brodcst dte, ctegory, RSS provder, brodcst chnne), owng users to serch cross the vbe dt. To fctte the resuts vsuston, the system provdes Web nterfce showng the rnked resuts. The dets of the browsng nterfce re presented n Secton..3.. Lngustc Anyss nd Query Budng In our system RSS feeds re used s the strtng pont for the ggregton process. Our system works wth both Itn nd 390 Journ of Dgt Informton Mngement Voume 8 Number 6 December 00

5 Engsh nguge feeds, usng n utomtc trnston servce bsed on the OpenLogos toos []. The RSS tems re nysed by TreeTgger [3], nguge ndependent prt-of-speech softwre tht tokenses the nput text (.e., the tems ttes nd descrpton phrses) nd tgs ech word of the text wth descrptve be, ccordng to key set of grmmr ctegores. Let π be n RSS tem descrbed by the tupe. The ngustc nyss modue f rst spts the text of the tem s tte nd descrpton phrses nto m ndependent sentences. Then, t mps such sentences nto vector of key/vue prs, π((k,v),...,(k,v) f,...,(k,v) m,...,(k,v) m ), where the keys re the words n the sentences, the vues re the correspondng grmmr ctegores, f nd re the number of words n the f rst (.e., the tem s tte) nd st sentence (.e. the st tem s descrpton phrse), respectvey. Bsed on ths vector, the query constructon process works n two steps. Frst, for ech sub-vector s π, sc query q s but seectng the words n s tgged s nouns. Then, compex query Q s generted, jonng the bsc queres s foows: Q : m ( m ) q (0) Equton (0) ssoctes hgher weghts to queres derved from sentences occurrng erer, n order to emphsse the tte nd the nt descrpton sentences. An exmpe of the query constructon process s shown n Fgure 3. Fgure 3. Exmpe of query constructon from ngustc nyss.. Cross-Mod Custerng for RSS Items nd News Stores Aggregton The tsk of RSS tems nd news stores ggregton s ddressed usng cross-mod custerng gorthm. The process s mpemented n four steps:. RSSF-DTV ff nty mtrx budng;. RSSF equvence mtrx dervton; 3. RSSF connectvty grph generton; 4. Mutmed dossers constructon nd ndexng. In the frst step, serch resuts vector s computed for ech RSS tem. Let W {} m eb π be the set of downoded RSS tems, nd T {} n v N j be the set of brodcsted news stores. For j ech RSS tem π W eb the correspondng compex query Q s unched on the news stores speech trnscrptons stored n the TV ndex structure. The output of Q s stored n the serch resuts vector S (S,..., S n ), where S j s the Lucene score of The Lucene score of query q for document d s defned n zones.pche.org/hudson/job/lucene-trunk/jvdoc//org/pche/ucene/ serch/smrty.htm the query Q to the speech trnscrpton of the news story N j. We c the score S j s the ffnty of the news story N j to the RSS tem π. A the serch resuts vectors re then rrnged n the rows of the ffnty mtrx A [s,..., s,..., s m ] T [0,] m n. The mtrx A cn be consdered s spce trnsformton opertor tht projects the RSS tems from the Web spce,.e. the text spce of the compex queres, to the brodcst domn,.e. the mutmed spce of the TV news stores. Once the ff nty mtrx hs been constructed, we compute the smrty between RSS tems expotng ther projecton n the brodcst domn (step ). The smrty between the coupe of RSS tems (π,π b ) W eb s evuted usng n symmetrc ffnty functon s (π,π b ), s def ned n the foowng equtons: ff ff ( s s ) S (, b π, π b) cos () s S π, π ) > α S( π, π ) > α then Eq, π ) ( s ( π () S π, π ) > α S( π, π ) α then Ent, π ) ( ( π (3) where s nd s b re the serch resuts vectors obtned n the f rst step nd cos(s, s b ) s the Cosne smrty between s nd s b. Intutvey, the functon S( ) n Equton () mesures how much the tem π s expned by the tem π b, n the spce of ther serch resuts vectors. Equton () defnes the semntc equvence reton between π nd π b, whe Equton (3) def nes the semntc entment reton from π nd π b. Notce tht the tter retonshp woud not be dscovered by usng the pn Cosne smrty mesure. For ech coupe of serch resuts vectors (s, s b ),b...m we compute the correspondng eement of the RSSF equvence mtrx m m, whose eements e b re defned s foows: e E R b S b (, π ) b : S( π, π ) π (4) 0 b otherwse b α Once E s ccuted, the prmry connectvty grph G (V,E) s but (step 3). Ech vertex v of G corresponds to n RSS tem. Two vertces v nd v re connected from v to v f the correspondng eement e b b E s greter thn α. The α-cut vue gurntees tht every pr of connected RSS tems (π,π ) hs semntc reevnce of t est α. Fgure 4 shows n exmpe of the constructon of the grph G for vrous vues of the threshod α. Strtng from G, we bud the set of dsconnected sub grphs d {d,...d M } G(step 4). For exmpe, from Fgure 4(), t woud be d {(π, π,π 3,π 4 ), (π 5,π 6 )} for α > 0.. Ech sub grph d corresponds to n ggregton of semntcy reted RSS tems nked ccordng to ther retonshps. Ech vertex of d s beed ccordng to the tte of the correspondng newspper rtce, nd the whoe sub grph s represented by the tte of the most representtve newspper rtce (.e., the tem s tte whose sum of the row eements of E s mxmsed). As n exmpe, Fgure 5 ustrtes n ggregton, tken from the ones ctuy detected by our prototype. As shown n the fgure, rrows between two boxes represent the dscovered vector smrty condton between the source box nd the trget box, s per the ntrnsc symmetrc nture of the smrty mesurement of Equton (). The representtve eement s the green one. Journ of Dgt Informton Mngement Voume 8 Number 6 December 00 39

6 Fgure 4. Exmpe of the equvence mtrx between the set of 6 RSS tems {π } nd the correspondng connectvty grph for three vues of α. Fgure 5. Exmpe of ggregton of semntcy reted RSS tems nked ccordng to ther retonshps. The mutmed dosser s f ny constructed by tkng, for ech tem π j d, j... d, the set of the news stores N k tht compy wth the foowng condton: s η (5) jk where s jk s the ff nty of the news story N k to the RSS tem π j (s computed n the f rst step of the custerng procedure), nd η s fxed threshod. For ech mutmed dosser, text document, whch ncudes both the RSS tems ttes nd descrpton phrses nd the news stores trnscrptons consttutng the dosser, s generted. Fny, such documents re ndexed by Lucene nd mde ccessbe through the MA repostory (see Fgure )..3 Mutmod Nvgton Servce The system supports both smpe queres (e.g., one or more serch keywords) s we s more dvnced queres (e.g., weghted queres, Booen opertors) for serchng nd retrevng the ggregtons. As smpe exmpe, Fgure 6 shows n exmpe for the query grbge AND Npes. To fctte the resuts vsuston, the system provdes browsbe Web pge showng the rnked resuts. For ech retreved dosser, the system not ony sts the bsc nformton,.e. the dosser s tte, score nd st updte tmestmp, nd t provdes the nks for the ncuded news stores nd newspper Fgure 6. Exmpe of the serch resuts Web pge for the query grbge AND Npes rtces. Addtony, s the serch resuts pge s provded s n RSS feed, users cn subscrbe to the submtted query, nd utomtcy receve notf cton when the resuts pge s modfed,.e. when ether n redy ncuded dosser s updted or new one s dscovered. 3. Experment Evutons Ths secton presents the experment evuton of the prts consttutng our prototype system. The system ws run for seven months, rngng from the end of November 007 to the begnnng of June 008. In ths perod of tme, we coected bout 88,80 onne rtces nd 3,940 news stores, resutng from the segmentton of 3,670 newscst progrmmes. The onne rtces were downoded from 95 RSS feeds supped by 6 onne newsppers nd press gences Web stes. The newscsts were cqured from the dy progrmmng of seven nton TV chnnes. We set α 0.8 n Equton (4) nd η 0.5 n Equton (5), resutng n tot of 4,87 mutmed dossers. 3. DTV Strem Processng Performnce The TV progrmme detecton nd segmentton tsk s performed cqurng ve strems from seven mjor nton chnnes 4 hours/dy nd 365 dys/yer. The system ebortes 6 progrmmes/dy concentrted round the mn edtons of the newscsts (round p.m. nd round 8 p.m.), resutng n bout 0 hours per dy of eborted udovsu mter. The tot eborton tme on verge s bout 3.74 tmes the progrmme durton, tht s normy newscst of hf n hour durton s serchbe nd retrevbe wth ts components fter ess thn hours fter ts end. We tested the progrmme detecton technque descrbed n Secton.. n two dstnct experments. In the f rst experment, dfferent reference cps hd to be dentf ed n dt set consttuted by 78 cps tken rndomy from dy teevson schedues. The second experment conssted n detectng the strtng nd endng jnges of 7 dstnct newscst progrmmes (tot 4 cps) n contnuous f ow of cqured udovsu mter. In the f rst experment, the mesured verge detecton ccurcy of the process ws 0.80, whe rec ws In the second experment reched precson ws.00, whe rec ws 0.90, consderng s detected tems ony progrmmes for whch both strtng nd endng jnges hd been dentf ed. 39 Journ of Dgt Informton Mngement Voume 8 Number 6 December 00

7 We tested the news story segmentton gorthm descrbed n Secton.. on bout 40 hours of mter, for whch true story boundres were mnuy dentf ed. To ssess the system performnce we used n gnment mesurement tkng nto ccount strtng boundres nd endng boundres wth dfferent weghts, s we s consderng mssng mter s hvng more mpct thn extr mter on the mesurement. The obtned verge precson nd rec were 0.76 nd 0.73, respectvey. 3. RSSF Strem Processng Performnce To test the effectveness of the RSSF strem processng chn, we set up poo of 5 expert users tken from the empoyees of our orgnston. Ech user ws sked to evute the consstency of the generted mutmed dossers. To ths end, we desgned nd mpemented n HTML Web nterfce, showng st of dossers, rndomy seected mong the vbe ones. For ech dosser, the foowng three mrkers were sked to be evuted, usng judgement sce from (poor) to 5 (exceent):. For the whoe dosser, ssgn semntc coheson ndex (I ) tht ref ects the over strength of the RSS nd brodcst news strems ggregton;. For ech ncuded RSS tem, ssgn consstency ndex (I ) to the concept expressed by the dosser; 3. For ech ncuded news story, ssgn consstency ndex (I 3) to the concept expressed by the dosser. In ddton, we sked to choose tte T from mong the ones beongng to the ggregted onne rtces, nd to evute the representtveness of the chosen tte (I 4) on the sme judgement sce, w.r.t. the concept expressed by the whoe dosser. The effectveness of the tte seecton strtegy ws then evuted by mens of the foowng ndces:. The rto between the number of correcty seected ttes (.e., the number of dossers for whch the system-chosen tte grees wth the user-chosen tte) nd the tot number of ssessed dossers (I 5);. The verge representtveness of the correcty seected ttes (I 6); 3. The verge representtveness of the wrongy seected ttes (I 7); 4. The verge representtveness of the user-seected ttes (I 8). The evutons were coected from the end of Mrch 008 to the begnnng of June 008, obtnng tot number of 65 evuted dossers. Tbe reports the vues obtned for the ndces I to I 8 n terms of verge, stndrd devton nd conf dence nterv. Over, the system shows n outstndng performnce, gettng n most cses score of ether 4 or 5 n most of the ndctors tht ws chosen for ssessment. Poor performnces for I 5 seem to ndcte tht the gorthm used by the system to choose the dosser s tte needs to be further mproved. However, s I 6 > I 8 nd σ 6 < σ 8, we cn stte tht the ttes chosen by the system re more representtve thn the others. Furthermore, I 8 I 7, ndctng tht when the tte utomtcy seected re wrong, they re st sgnf cnty reevnt to the concept expressed by the dosser. I I I 3 I 4 I 5 I 6 I 7 I 8 Vue Std. Dev n C.. strt n C.. end n Tbe. Accurcy ndces of the mutmod ggregton servce 4. Concusons Eff cent rge-sce mpementton of mutmed ndexng systems s n hrd tsk, snce opertng condtons of re systems my put very strngent requrements on the over performnces, nd re-fe envronments re often qute dversf ed w.r.t. prototype ones. As users my wnt to browse through rge mount of news contents wthout spendng much tme n serchng for the desred nformton, mutmed news mngement ppctons must provde serch nd retrev servces n tmey nd eff cent mnner. To ths end, ths rtce presented nd dscussed the mpementton of rge-sce utomtc cquston, segmentton nd ndexng of brodcst news content. The over system performnces re encourgng, nd the ccurcy of the progrmme boundry detecton nd segmentton re t good eve of mturty. In ddton, ths rtce ntroduced nove method for ggregton of mutmod sources of nformton, bsed on semntc reevnce functon ctng s kerne to dscover the semntc ff ntes of heterogeneous nformton tems, nd on n vector projecton smrty prncpe on whch semntc dependency grphs mong nformton tems cn be constructed. The generty of the proposed method ows ts ppcton to wde coecton of re cses, nd n ths rtce we presented n ppcton of t n the re of mutmod news ggregton. We deveoped prototyp system, whch hs been evuted by coectng dy teevson newscsts from 7 mjor Itn brodcsters nd 95 RSS news feeds from onne webstes for perod of bout 8 months. Obtned resuts re very encourgng nd demonstrte the robustness nd effectveness of our method. Future deveopments w regrd both the theoretc foundtons nd the prctc outcomes of our technoogy. From the theoretc perspectve, modes of ntegrton of more thn two nformton tems sets w be nvestgted. On the mpementton sde, to further ensure fu-potent expotton of our system n ndustr contexts, future work w so regrd both the mprovement of the gener performnces n terms of eborton tency nd n terms of the ccurcy of most crtc processng bocks of the system, bsed on the performnces presented n ths work. References [] RSS Specf ctons, ctons.com/ [] OpenLogos Mchne Trnston, [3] TreeTgger nguge ndependent prt-of-speech tgger, [4] Brugnr, F., Cettoo, M., Federco, M., Gun, D. (000). A system for the segmentton nd trnscrpton of Itn rdo news. In Proc. of RIAO, Content-Bsed Mutmed Informton Access, Sept Journ of Dgt Informton Mngement Voume 8 Number 6 December

8 [5] Bs, R., Cmms, M., Dont, E. (005). Rtrover: A web ppcton for semntc ndexng nd hypernkng of mutmed news. In Proc. of Internton Semntc Web Conference, pges 97-, June 005. [6] Bnerjee, S., Rmnthn, K., Gupt, A. (007). Custerng short texts usng Wkped. In ACM SIGIR Conf. on Reserch nd Deveopment n Informton Retrev, Juy 007. [7] Chu, T., Chng, S., Chsorn, L., Hsu, W. (004). Story boundry detecton n rge brodcst news vdeo rchves: Technques, experence nd trends. In ACM Mutmed 004, Oct [8] De Snto, M., Percnne, G., Snsone, C., Vento, M. (006). Unsupervsed news vdeo segmentton by combned udo-vdeo nyss. In: Proc. of the Int. Work. on Mutmed Content Representton, Cssf cton nd Securty (MRCS), pges 73-8, Sept [9] Hosh., K. (004). Shot boundry determnton on MPEG compressed domn nd story segmentton experments for TRECVID 004. In Proc. Of TRECVID Workshop 004, Feb [0] Hsu, W., Chngy, S.F., Hungy, C.W., Kennedyy, L., Lnz, C.Y., Iyengr, G. (004) Dscovery nd fuson of sent mutmod fetures towrds news story segmentton. In Proc. of Storge nd Retrev Methods nd Appctons for Mutmed, pges 44-58, Jn [] Hung, W., Webster, D. (004). Integent RSS news ggregton bsed on semntc context. In ACM SIGIR Workshop on Informton Retrev n Context, pges 40-4, Juy 004. [] Krj, W., Smeton, A., Over, P. (004) TRECVID 004: An overvew. In Proc. of TRECVID Workshop 004, Feb [3] L, X., Yn, J., Deng, Z., J, L., Fn, W., Zhng, B., Chen. Z. (007). A nove custerng-bsed RSS ggregtor. In 6th Int. Conf. on Word Wde Web, pges , My 007. [4] Deègse, P., Estève, Y., Megner, S., Mern, T. (005). The LIUM speech trnscrpton system: CMU Sphnx III-bsed system for french brodcst news. In In Proc. of Interspeech 05, Sept [5] Pckerng, M.J., Wong, L., Rueger, S.M. (003). Anses: Summrston of news vdeo. In Proc. of Internton Conference on Imge nd Vdeo Retrev (CIVR), pges , Jn [6] Quènot, G. M., Mrru, D., Ayche, S., Chrhd, M., Bescer, L. (004). CLIPS-LIS-LSR-LABRI experments t TRECVID 004. In Proc. of TRECVID Workshop 004, Feb [7] Schmd, H. (994) Probbstc Prt-of-Speech Tggng Usng Decson Trees. In Proc. of the Int. Conf. on New Methods n Lnguge Processng, Sept [8] Tmur, H., Mor, S., Ymwk, T. (978). Texture fetures correspondng to vsu percepton. IEEE Trns. on Systems, Mn nd Cybernetcs, 8(6): , 978. [9] Vokmer, T., Thhogh, S.M.M., Wms, H.E. (004). RMIT unversty t TRECVID 004. In Proc. of TRECVID Workshop 004, Feb [0] Xu, C., Wng, J., Lu, H., Zhng, Y. (008). A Nove Frmework for Semntc Annotton nd Personzed Retrev of Sports Vdeo. IEEE Trnsctons on Mutmed, 0(3):4-436, Apr 008. [] Xu, H., Chu, T.S. (006). Fuson of AV fetures nd extern nformton sources for event detecton n tem sports vdeo. ACM Trns. Mutmed Comput. Commun. App., ():44-67, Feb [] Zh, Y., Cho, X., Zhng, Y., Jved, O., Ymz, A., Rf, F. (004). Unversty of Centr Ford t trecvd 004. In Proc. of TRECVID Workshop 004, Feb [3] Zhng, Y., Xu, C., Ru, Y., Wng, J., Lu, H. (007). Semntc event extrcton from bsketb gmes usng mut-mod nyss. In IEEE Int. Conf. On Mutmed nd Expo, pges 90-93, Juy 007. [4] Montgnuoo, M., Messn, A. (009). Pre Neur Networks for Mutmod Vdeo Genre Cssfcton. In Mutmed Toos nd Appctons, 4():5-59, Jn [5] Apche Lucene project, [6] Bhttchryy, A. (943). On mesure of dvergence between two sttstc poputons def ned by ther probbty dstrbutons. Buetn of the Ccutt Mthemtc Socety 35: Journ of Dgt Informton Mngement Voume 8 Number 6 December 00

9 Authors Bogrphes Dr. Murzo Montgnuoo receved hs Lure degree n Teecommunctons Engneerng from the Poytechnc of Turn n 004, fter deveopng hs thess t the RAI Reserch Centre. Currenty. In 008 he receved hs Ph.D. n Busness nd Mngement t the Unversty of Turn, n coborton wth RAI, nd supported by EurX S.r.., Turn. Hs nt contrbutons to computer scence were n the re of rtf c ntegence, specf cy n the semntc cssf cton of udovsu content. In prtcur, he ws nvoved n reserch projects on utomtc cssf cton nd chrcterston of teevson genre. Hs current reserch nterests re mosty ddressed n the context of Web dt mnng nd mutmed dt mnng. He s so co-uthor of sever scentf c works pubshed on Journs or n Internton Conferences n ths subject re. Aberto Messnegn hs coborton s reserch engneer wth RAI n 996, when he competed hs MS Thess n Eectronc Engneerng (t Potecnco d Torno) bout objectve quty evuton of MPEG vdeo codng. After strtng hs creer s desgner of RAI s Mutmed Ctogue, he hs been nvoved n sever ntern nd nternton reserch projects n the f ed of dgt rchvng, wth prtcur emphss on utomted documentton, nd utomted producton. Hs current nterests re rngng from f e formts nd metdt stndrds to the domn of content nyss nd nformton extrcton gorthms, where he now concentrtes hs mn focus. Recenty, he hs strted promsng reserch ctvtes concernng semntc nformton extrcton from the numerc nyss of udovsu mter, prtcury n the f ed of conceptu chrcterston of mutmed objects, genre cssf cton of mutmed tems, utomtc edtor segmentton of TV progrmmes. He s so uthor of technc nd scentf c pubctons n ths subject re. He hs extensve cobortons wth the oc Unversty of Torno Computer Scence Deprtment, whch ncude common reserch projects nd students tutorshp. To compete hs scentf c formton, he hs recenty decded to tke PhD n the re of Computer Scence. He s ctve member of sever EBU projects ncudng P/TVFILE, P/MAG nd P/CP, chrmn of the P/SCAIE project deng wth utomtc metdt extrcton technques. He s currenty workng n the EU PrestoSpce project n the Metdt Access nd Devery re. He hs served s Progrmme Commttee Member n Spec Trck of the 0 th Conference of Itn Assocton of Artf c Integence, nd n the Workshop on Ambent med Devery nd Interctve Teevson. Roberto Borgoto grduted n Teecommuncton Engneerng t Potecnco d Torno n 999. Snce 00, he hs been workng for RAI - Rdoteevsone Itn t the R&D deprtment (Centro Rcerche e Innovzone Tecnoogc) n Turn. Inty, he ws nvoved n sever projects grvttng round the RAI mutmed ctogue, ced CMM, regrdng metdt ngeston nd trnsformton. More recenty, Mr Borgoto hs been workng n tem tht s deveopng n utomtc metdt extrcton ptform whch s ctuy used extensvey n RAI for experment purposes, nd even for re n the producton evronment. Hs mjor professon nterests re metdt nd essence trnsformton, system ntegrton nd workf ow mngement. Journ of Dgt Informton Mngement Voume 8 Number 6 December