Information Filtering and Retrieval lecture SS 2007

Transcription

1 Infomation Filteing and Retieval lectue SS 7 D. Dominik Kuopka Pof. D. Mathias Weske Repetition: limitations of models without tem intedependency Real Wold usage, poblems appea Mophology? Synonymy? adaptation of the wold document pepocessing: Stopwod List Stemming Synonymy Substitution Adapted Real Wold usage on the adapted eal wold Unsolved: Homogaphy Metonymy Hyponymy Meonymy Dominik Kuopka 7 5-

2 Repetition: Main Idea of s with immanent tem intedependencies Tems can be pai-wise intedependent Tem dependency is coupled with the model Gade of intedependency is deived fom the document base using a co-occuence based method Example fo a simple co-occuence based method: compute occus 45 times in document base linux occus 98 times in document base compute and linux occus 76 times in document base > compute and linux ae highly intedependent Dominik Kuopka 7 5- Repetition: GVSM Document Vectos Mintem Vectos Tem Vectos Dominik Kuopka 7 5-4

3 Repetition: GVSM pos and cons Pos Similaities anges between and Tem Vectos do not need to be othogonal, angle depends on co-occuence of tems within the document base Cons Moe costly than VSM No diect suppot fo opeatos like AND/OR Dominik Kuopka Repetition: SANN Topology Quey Tems Document Tems Documents τ τ a τ b τ c t w q,t : w d,t : t a t b t c d d a d b τ #T t #T d #D Dominik Kuopka 7 5-6

4 Repetition: SANN Pos and Cons Pos No tem independency assumption Co-occuence of tems within the document base used fo activation of othe documents (feedback) Numbe of feedback cycles adaptable Usable fo document similaities Cons No diect suppot fo opeatos like AND/OR If feedback cycles ae used > moe costly than VSM Dominik Kuopka This Lectue Advanced Latent Semantic Index Geneal poblems of models with immanent tem intedependencies Dominik Kuopka

5 Oveview on IF&IR models tem chaacteistics mathematical foundation models without tem intedependencies models with tem intedependencies immanent tem intedependencies tanscendent tem intedependencies set theoetic models Standad Boolean Fuzzy Set algebaic models Vecto Space Extended Boolean Genealized Vecto Space Latent Semantic Index Spead. Activation Neuonal Netwok Topic-based Vecto Space Backpop. Neuonal Netwok Enhanced Topic-based Vecto Space pobabilistic models Binay Independence Retieval Infeence Netwok Belief Netwok Language Retieval by Logical Imaging Dominik Kuopka Latent Semantic Index (LSI) Boowed fom the VSM: Idea: Repesent documents as vectos in a vecto space. Tems (initial) dimensions of the vecto space New Reduction of dimensions by Singula Value Decomposition and Rank eduction ( lossy compession of data) #(final) dimensions < #Tems Dominik Kuopka 7 5-5

6 6 Dominik Kuopka 7 5- Excusion on Matices Sample Matix:,,,, m m m m M Matix-Vecto-Multiplication: +,,,, m m v m m v v M Sample Vecto: v v v Dominik Kuopka 7 5- Stetches ,, v v v S.8.4,, v S Sv S v Stetches expand the space and ae always diagonal matices: s n s s S L M O M L

7 Pepfames p x Pepfames ae the basic dimension vectos of a otated coodinate system. p y - α6 p x p p x, x,, p y p p y, y, - p x, p p p x, y, y, cos( α) sin( α) sin( α) Dominik Kuopka 7 5- Hanges α6 v,, v v px, py, H, α 6 px, py, Hanges hangs the system to a pepfame system. -.5 H v,.87.87,.5 Hv Hv Dominik Kuopka

8 Alignes v,, v v T.8.4 px, py, px, px, A, α 6 px, py, py, py, - α6 Alignes align a pepfame system to the coodinate system. -.5 Av,.87.87,.5 Av Av.5.89 Dominik Kuopka Singula Value Decomposition (SVD) Theoem: Evey Matix M can be decomposed into a Hange H, Stetche S and Aligne A. M H S A Sample: Stetche values ae also named as singula values of M Moe: Dominik Kuopka

9 Singula Values Show how much each dimension is used by a matix High pos. o neg. value: dimension has high impact on matix stuctue Value nea zeo: dimension has low impact on matix stuctue singula value Typical distibution of singula values (invese odeing) Dim. numbe Dominik Kuopka SVD-Compession Oiginal M Lossy compessed M Reduce dimensions by emoving small singula values obsolete data Hint: sample has to less dimensions to show eduction of data to stoe. Dominik Kuopka

10 SVD Image Compession Sample Oiginal SVD Compessed Souce: Dominik Kuopka Document-Tem Matix Latent Semantic Index Singula Value Decomposition Reduction of dimensions (lossy compession) Calculation of pai-wise similaities between all documents Dominik Kuopka 7 5-

11 Sample Docs + Queies Document-Tem Matix M Dominik Kuopka 7 5- Sample: SVD Hange Stetche Aligne T M Dominik Kuopka 7 5-

12 Sample: SVD Hange Stetche Aligne T M Dominik Kuopka 7 5- Sample: SVD Hange Stetche Aligne T M Dominik Kuopka 7 5-4

13 Sample: SVD + compession Hange Stetche Aligne T M X-Y Axis ae swapped M Dominik Kuopka Sample: Compaison of Matices M human inteface compute use system esponse time EPS suvey tees gaph minos LSI uses tem co-occuences to ise o lowe tem occuence in a specific document if this document is (diectly o indiectly) simila to othe documents. human inteface compute use system M esponse time EPS suvey tees gaph minos Dominik Kuopka 7 5-6

14 Sample Similaities Using MM T : sim(c, m4) sim(c, m4) sim(c5, m4) Using M M T : sim(c, m4) -, sim(c, m4), sim(c5, m4),68 Dominik Kuopka LSI: Pos and Cons Pos Similaities ae floats (nomalization is possible) Sophisticated evaluation of diect and indiect tem co-occuences by SVD lossy dimension eduction Seveal weighting possibilities fo tems: Plain occuence, tf-idf, etc Cons Moe costly than VSM, SVD is ticky No diect suppot fo opeatos like AND/OR Dominik Kuopka

15 Oveview on IF&IR models tem chaacteistics mathematical foundation models without tem intedependencies models with temintedependencies immanent tem intedependencies tanscendent tem intedependencies set theoetic models Standad Boolean Fuzzy Set Tem similaities ae deived fom diect o indiect co-occuences of tems within the document base. algebaic models pobabilistic models Vecto Space Question: Ae these similaities coect? Binay Independence Retieval Infeence Netwok Extended Boolean Belief Netwok Language Genealized Vecto Space Latent Semantic Index Spead. Activation Neuonal Netwok Topic-based Vecto Space Backpop. Neuonal Netwok Retieval by Logical Imaging Enhanced Topic-based Vecto Space Dominik Kuopka Linguistic point of view: expected similaities (/) Inflection (e.g. house, houses) Little change of meaning > sim -> vey high Synonymy (e.g. automobile, ca) Same meaning in a specific context > sim -> vey high Hyponymy (is-a elationship) Should depend on the numbe of intemediate subodinate steps > sim -> high Meonymy (pat-of elationship) Should depend on the numbe of intemediate subodinate steps > sim -> high Dominik Kuopka 7 5-5

16 Linguistic point of view: expected similaities (/) Composition (maste + mind mastemind) English: seldom, meaning usually cannot be deived diectly fom wods > sim -> low Geman: Well defined is-a elationship (Hyponymy) > sim -> high Wod goup ( New Yok vs. new and Yok ) Usually no diect elationship between wods, especially fo names of pesons o places. > sim -> vey low No linguistic phenomenon at all > sim null Dominik Kuopka 7 5- Test: Do linguistic similaities match simple co-occuence based similaities? Document Base English Wikipedia Snapshot taken at 4 th of July 4 745,546 individual documents Geman Wikipedia Snapshot taken at 4 th of July 4 6,5 individual documents Co-occuence based measues Jaccad Dice Cosine Dominik Kuopka 7 5-6

17 Co-occuences of English Wikipedia (except) a b Jaccad Dice Cosine Phenomenon exp. Sim. New Yok,45,94,484 wod goup vey low ambe necta,5,, wod goup vey low ca tee,,4,4 none null house ed,59,, none null mastemind maste,4,8,5 composition low mastemind mind,,7,5 composition low wheel ca,6,5,5 meonymy high body leg,,45,8 meonymy high plant tee,8,48,49 hyponymy high plant tulip,5,,5 hyponymy high ca automobile,4,49,78 synonymy vey high hope espeance,,,8 synonymy vey high house houses,7,5,79 inflection vey high mouse mice,8,95,6 inflection vey high Dominik Kuopka 7 5- Co-occuences of Geman Wikipedia (except) a b Jaccad Dice Cosine Phenomenon exp. Sim. New Yok,57,78,746 wod goup vey low Bill Gates,75,4,98 wod goup vey low Auto Baum,5,5,5 none null Haus ot,,6,6 none null Gatenzweg Zweg,,5,45 composition high Gatenzweg Gaten,,, composition high Reifen Auto,8,5,4 meonymy high Köpe Fuß,8,5,8 meonymy high Pflanze Baum,59,, hyponymy high Pflanze Tulpe,,, hyponymy high Auto Automobil,79,46,7 synonymy vey high Fahstuhl Lift,,45,47 synonymy vey high Haus Häuse,6,7,46 inflection vey high Maus Mäuse,84,55,86 inflection vey high Dominik Kuopka

18 Linguistic Substantiation (/) Inflection Usually only a few inflected foms of the same wod ae needed in a text. > co-occuences of inflected foms ae low, but high similaity is expected Synonymy In naative texts synonymy is quite often > good chances fo high co-occuences In scientific texts synonymy is often avoided > bad chances fo high co-occuences (while high similaity is expected) Dominik Kuopka Linguistic Substantiation (/) Hyponymy / Meonymy It is not usual to enumeate all subodinates o pats of things in texts (except fo constuction manuals o ontology elated documents) > co-occuences tend to be low (while high similaity is expected) Composition English: seems to be good epesented by co-occuence based measues (low co-occuences while low similaity is expected) Geman: Same poblem as fo Hyponymy. (low co-occuences while high similaity is expected) Dominik Kuopka

19 Linguistic Substantiation (/) Wod Goup Fo often used wod goups co-occuences tend to be high (while low similaity is expected) No linguistic phenomenon at all In cases whee wod combinations ae often used fo some abitay eason, they tend to have high co-occuence (while low similaity is expected) Dominik Kuopka Summay Simple co-occuence based methods (e.g. like used by GVSM) fo estimation of tem intedependencies / similaities Tend to undeestimate tem similaities in geneal Delive scewed tem similaities in some cases Futhe eseach is needed to evaluate moe sophisticated co-occuence methods like LSI, but the poblems seem to be fundamental. Dominik Kuopka

20 Next Lectue s with tanscendent tem intedependencies Fuzzy Set Backpopagation Neuonal Netwok Retieval by Logical Imaging Evaluation of s Dominik Kuopka 7 5-9