Generic Text Summarization

Size: px
Start display at page:

Download "Generic Text Summarization"

Transcription

1 June 27, 2012

2 Outline Introduction 1 Introduction Notation and Terminology

3 Text Summarization Introduction Notation and Terminology Two Types of Text Summarization Query-Relevant Summarization: Specific to information retrieval applications, for example the snippets below each result returned by a search engine. Attempts to summarize the information a document contains pertaining to specific search terms. Answers the question "what does this document say about <query>?" Generic Summarization: Provides an overall summary of all the information contained in a document. Answers the question "what is this document about?"

4 Notation and Terminology The Goal of Generic Summarization A document is expected to contain several topics. Main topics are discussed extensively by many sentences. Minor topics have less sentence support and exist to support the main topics. The specific goal of generic summarization is as follows: Objective: For a given document with n sentences, choose k sentences (as specified by the user) from the document that best describe the main topics of the document. Keep redundancy of the chosen sentences to a minimum. This paper describes two methods to achieve this objective.

5 Notation and Terminology Text Data: The Term-Document Matrix As is common with text-mining applications, our original data structure is an m n matrix of term frequencies. Each column of the matrix represents a passage of text. A passage could be defined as an entire document, a paragraph, or a sentence depending on the application. For our purposes, each passage will represent a sentence. A m n = Term 1 Term i Term m Sent Sent. j...sent. n f ij

6 Term-Weighting Schemes Notation and Terminology Generally term-weighting is used to downplay the effect of frequently used words, while enhancing the effect of rare but semantically important words. A weighted term-document matrix contains entries W ij = L(f ij ) G(i) where L and G represent local and global weights respectively.

7 Term-Weighting Schemes Notation and Terminology Generally term-weighting is used to downplay the effect of frequently used words, while enhancing the effect of rare but semantically important words. A weighted term-document matrix contains entries W ij = L(f ij ) G(i) where L and G represent local and global weights respectively. Local Term Weighting: Determined by the use of the term within the sentence (or document). Binary: L(f ij ) = 1 if f ij 1, 0 otherwise Logarithmic: L(f ij ) = log(1 + f ij )

8 Term-Weighting Schemes Notation and Terminology Generally term-weighting is used to downplay the effect of frequently used words, while enhancing the effect of rare but semantically important words. A weighted term-document matrix contains entries W ij = L(f ij ) G(i) where L and G represent local and global weights respectively. Local Term Weighting: Determined by the use of the term within the sentence (or document). Binary: L(f ij ) = 1 if f ij 1, 0 otherwise Logarithmic: L(f ij ) = log(1 + f ij ) Global Term Weighting: Determined by the use of the term within the document (or collection). IDF: G(i) = log(n/n(i)) where N is # of sentences and n(i) is # sentences containing term i.

9 Term-Weighting Schemes Notation and Terminology Generally term-weighting is used to downplay the effect of frequently used words, while enhancing the effect of rare but semantically important words. A weighted term-document matrix contains entries W ij = L(f ij ) G(i) where L and G represent local and global weights respectively. Local Term Weighting: Determined by the use of the term within the sentence (or document). Binary: L(f ij ) = 1 if f ij 1, 0 otherwise Logarithmic: L(f ij ) = log(1 + f ij ) Global Term Weighting: Determined by the use of the term within the document (or collection). IDF: G(i) = log(n/n(i)) where N is # of sentences and n(i) is # sentences containing term i. Normalization: A passage vector A i is normalized by its length A i.

10 Decompose document into sentences. All sentences form the candidate set S. Create weighted term-frequency vectors, A i for each sentence, using the dictionary of terms from the entire document (long, sparse vectors). Create weighted term-frequency vector, D for the entire document.

11 Decompose document into sentences. All sentences form the candidate set S. Create weighted term-frequency vectors, A i for each sentence, using the dictionary of terms from the entire document (long, sparse vectors). Create weighted term-frequency vector, D for the entire document. For each sentence A i S compute relevance score between A i and D as the inner product A T i D.

12 Decompose document into sentences. All sentences form the candidate set S. Create weighted term-frequency vectors, A i for each sentence, using the dictionary of terms from the entire document (long, sparse vectors). Create weighted term-frequency vector, D for the entire document. For each sentence A i S compute relevance score between A i and D as the inner product A T i D. Select sentence A j with highest relevance score and add it to the summary.

13 Decompose document into sentences. All sentences form the candidate set S. Create weighted term-frequency vectors, A i for each sentence, using the dictionary of terms from the entire document (long, sparse vectors). Create weighted term-frequency vector, D for the entire document. For each sentence A i S compute relevance score between A i and D as the inner product A T i D. Select sentence A j with highest relevance score and add it to the summary. Delete A j from S and eliminate all terms contained in A j from the document. Recompute weighted term-frequency vector D for the document.

14 Decompose document into sentences. All sentences form the candidate set S. Create weighted term-frequency vectors, A i for each sentence, using the dictionary of terms from the entire document (long, sparse vectors). Create weighted term-frequency vector, D for the entire document. For each sentence A i S compute relevance score between A i and D as the inner product A T i D. Select sentence A j with highest relevance score and add it to the summary. Delete A j from S and eliminate all terms contained in A j from the document. Recompute weighted term-frequency vector D for the document. Continue until k sentences are chosen for the summary.

15 is an information retrieval technique that employs the Singular Value Decomposition (SVD) to create a lower dimensional semantic space where related words are mapped near each other.

16 is an information retrieval technique that employs the Singular Value Decomposition (SVD) to create a lower dimensional semantic space where related words are mapped near each other. Singular Value Decomposition For each A R m n of rank R, there are orthogonal matrices U m m = [u 1 u 2... u m] and V n n = [v 1 v 2... v n] and a matrix S R R = diag(σ 1, σ 2,..., σ R ) padded with zeros to make an m n rectangular matrix so that ( S 0 A = U 0 0 ) V T = R σ i u i v T i with σ 1 σ 2... σ R > 0 i=1

17 is an information retrieval technique that employs the Singular Value Decomposition (SVD) to create a lower dimensional semantic space where related words are mapped near each other. Singular Value Decomposition For each A R m n of rank R, there are orthogonal matrices U m m = [u 1 u 2... u m] and V n n = [v 1 v 2... v n] and a matrix S R R = diag(σ 1, σ 2,..., σ R ) padded with zeros to make an m n rectangular matrix so that ( S 0 A = U 0 0 ) V T = R σ i u i v T i with σ 1 σ 2... σ R > 0 i=1 In light of this, we can express each column (sentence) of the matrix A algebraically as a linear combination of right singular vectors as follows: A i = V T 1i (σ 1u 1 ) + V T 2i (σ 2u 2 ) + + V T Ri (σ Ru R )

18 Change of Basis Introduction Lets try to understand this lower dimensional space with a simple example. Any vector should be thought of as coordinates in a basis. Typically, we think of points using the standard" basis that is used (at least in 3 dimensions) in any calculus class.

19 Change of Basis Introduction Lets try to understand this lower dimensional space with a simple example. Any vector should be thought of as coordinates in a basis. Typically, we think of points using the standard" basis that is used (at least in 3 dimensions) in any calculus class. (2, 3, 4) = 2î, 3ĵ, 4ˆk =

20 Change of Basis Introduction Lets try to understand this lower dimensional space with a simple example. Any vector should be thought of as coordinates in a basis. Typically, we think of points using the standard" basis that is used (at least in 3 dimensions) in any calculus class. (2, 3, 4) = 2î, 3ĵ, 4ˆk = Our original sentence/passage vectors can be considered as points/coordinates in the same way, only in much larger dimensions. Each term in the dictionary can be thought of as a standard basis vector. Term 1 0 T i =. Term i. Term m

21 Change of Basis Introduction So our term-frequency vector is simply a linear combination of these standard-basis term-vectors, T i. Term 1 f 1j f 2j.. A j = = f Term i f ij 1j T 1 + f 2j T f mj T m.. Term m f mj

22 Change of Basis Introduction So our term-frequency vector is simply a linear combination of these standard-basis term-vectors, T i. Term 1 f 1j f 2j.. A j = = f Term i f ij 1j T 1 + f 2j T f mj T m.. Term m f mj The SVD gives us a NEW representation of our data in a lower-dimensional space. The entries in the columns of V T give us the coordinates of each document in the space spanned by the columns of U, scaled by the singular values (σs). A i = V T 1i (σ 1u 1 ) + V T 2i (σ 2u 2 ) + + V T Ri (σ Ru R )

23 Change of Basis Introduction So our term-frequency vector is simply a linear combination of these standard-basis term-vectors, T i. Term 1 f 1j f 2j.. A j = = f Term i f ij 1j T 1 + f 2j T f mj T m.. Term m f mj The SVD gives us a NEW representation of our data in a lower-dimensional space. The entries in the columns of V T give us the coordinates of each document in the space spanned by the columns of U, scaled by the singular values (σs). A i = V T 1i (σ 1u 1 ) + V T 2i (σ 2u 2 ) + + V T Ri (σ Ru R ) The theory tells us that the k-truncated SVD, A i = V T 1i (σ 1u 1 ) + V T 2i (σ 2u 2 ) + + V T ki (σ k u k ) gives us the closest rank k approximation to A with regard to Euclidean distance.

24 We began with m extremely sparse basis vectors, containing exactly one nonzero entry. We reduced the dimension of the space to k using a truncated SVD, and the result was a new set of k basis vectors, {σ i u i }, containing both positive and negative values.

25 We began with m extremely sparse basis vectors, containing exactly one nonzero entry. We reduced the dimension of the space to k using a truncated SVD, and the result was a new set of k basis vectors, {σ i u i }, containing both positive and negative values. This is a projection into a lower dimensional space. The major claims of Latent Semantic Indexing say that this lower dimensional space reveals semantic connections that were latent" in the original vector space. Such claims include: Original term space broken into k linearly independent components which identify major topics.

26 We began with m extremely sparse basis vectors, containing exactly one nonzero entry. We reduced the dimension of the space to k using a truncated SVD, and the result was a new set of k basis vectors, {σ i u i }, containing both positive and negative values. This is a projection into a lower dimensional space. The major claims of Latent Semantic Indexing say that this lower dimensional space reveals semantic connections that were latent" in the original vector space. Such claims include: Original term space broken into k linearly independent components which identify major topics. If a word combination is salient and recurring in a document, this pattern will be captured by one of the singular vectors.

27 We began with m extremely sparse basis vectors, containing exactly one nonzero entry. We reduced the dimension of the space to k using a truncated SVD, and the result was a new set of k basis vectors, {σ i u i }, containing both positive and negative values. This is a projection into a lower dimensional space. The major claims of Latent Semantic Indexing say that this lower dimensional space reveals semantic connections that were latent" in the original vector space. Such claims include: Original term space broken into k linearly independent components which identify major topics. If a word combination is salient and recurring in a document, this pattern will be captured by one of the singular vectors. New space is capable of capturing words that are related semantically. For example, the words physician and doctor will cooccur with other related words like hospital, nurse, and medicine and as a result the two words physician and doctor will be mapped near to each other in the singular vector space.

28 We began with m extremely sparse basis vectors, containing exactly one nonzero entry. We reduced the dimension of the space to k using a truncated SVD, and the result was a new set of k basis vectors, {σ i u i }, containing both positive and negative values. This is a projection into a lower dimensional space. The major claims of Latent Semantic Indexing say that this lower dimensional space reveals semantic connections that were latent" in the original vector space. Such claims include: Original term space broken into k linearly independent components which identify major topics. If a word combination is salient and recurring in a document, this pattern will be captured by one of the singular vectors. New space is capable of capturing words that are related semantically. For example, the words physician and doctor will cooccur with other related words like hospital, nurse, and medicine and as a result the two words physician and doctor will be mapped near to each other in the singular vector space. The magnitude of the singular value associated with each singular vector indicates the importance of this topic in the document, therefore the first k singular vectors are expected to contain the k most pertinent topics.

29 Construct the terms by sentences matrix A for the document D.

30 Construct the terms by sentences matrix A for the document D. Perform the SVD on A to obtain the matrix of right-singular vectors V T. Each column of V T gives the coordinates of a single sentence in the k dimensional LSI space. Therefore, the first row of V T gives the coordinates of all sentences along the first LSI basis component (i.e. the most important topic" contained in the document D). And similarly the j th row would give the coordinates of each sentence along the j th basis component.

31 Construct the terms by sentences matrix A for the document D. Perform the SVD on A to obtain the matrix of right-singular vectors V T. Each column of V T gives the coordinates of a single sentence in the k dimensional LSI space. Therefore, the first row of V T gives the coordinates of all sentences along the first LSI basis component (i.e. the most important topic" contained in the document D). And similarly the j th row would give the coordinates of each sentence along the j th basis component. For each row of V T take the sentence corresponding to the index of the largest value in the row and add it to the summary.

32 Construct the terms by sentences matrix A for the document D. Perform the SVD on A to obtain the matrix of right-singular vectors V T. Each column of V T gives the coordinates of a single sentence in the k dimensional LSI space. Therefore, the first row of V T gives the coordinates of all sentences along the first LSI basis component (i.e. the most important topic" contained in the document D). And similarly the j th row would give the coordinates of each sentence along the j th basis component. For each row of V T take the sentence corresponding to the index of the largest value in the row and add it to the summary. In other words, for each basis component, i = 1... k choose the sentence which has the largest coordinate along that component.

33 Two months of the CNN Worldview news programs. The evaluation database consists of closed captions of 549 news stories whose lengths are in the range of 3 to 105 sentences. Eliminated stories shorter than 10 sentences (why summarize something that is any shorter?) resulting in 243 documents.

34 The authors used a unique way to measure the performance of their summarizers. They asked 3 human evaluators to peruse the documents and choose 5 sentences that best summarized each document. Each document then had 5-15 sentences chosen by humans (depending on whether the 3 evaluators chose the same sentences). The disparity of the result surprised the authors! Each document had on average 9 sentences chosen by at least one evaluator, and among these 9 sentences only 1.2 sentences on average received a unanimous vote by all 3 evaluators! Even when sentence selection done by majority vote, still only a 2.5 sentence overlap on average. Disparities became even larger with longer documents. Decided to evaluate the automatic summarizers by comparing them to the 3 individual human summarizers, as well as the combined result decided by a majority vote. Computed Precision and Recall along with F-measure in both instances.

35 Precision: Recall: F-measure: P = S man S sum S sum R = S man S sum S man F = 2RP R + P The authors also examined the impact of various term-weighting schemes on the performance of their methods.

36

37 Term Weighting Results - Normalization Bad for SVD!!!

38 In "Summary" :) I used the relevance measure approach to summarize todays paper: Query-relevant text summaries are useful for answering such questions as whether a given document is relevant to the user s query, and if relevant, which part(s) of the document is relevant. The first summarizer (the one using the relevance measure) takes the sentence that has the highest relevance score with the document as the most important sentence, while the second summarizer (the one based on the latent semantic analysis) identifies the most important sentence as the one that has the largest index value with the most important singular vector. Consider the following passage taken from a CNN news story reporting the recent Israeli-Palestinian conflicts, political efforts for restoring the calm in the region, and hostile sentiments among Palestinian people: This paper presented two text summarization methods that create generic text summaries by ranking and extracting sentences from the original documents. For experimental evaluations, a database consisting of two months of the CNN Worldview news programs was constructed, and performances of the two summarization methods were evaluated by comparing the machine generated summaries with the manual summaries created by three independent human evaluators. However, when evaluated using evaluator 2 s results, the performance scores drop by 10% in average, dramatically dragging down the performance scores for the evaluation by a majority vote.

39 In "Summary" :) And from the SVD approach, Query-relevant text summaries are useful for answering such questions as whether a given document is relevant to the user s query, and if relevant, which part(s) of the document is relevant. (chosen twice by SVD) Finally, the causes of the large disparities in the evaluators manual summarization results are investigated, and discussions on human text summarization patterns are presented. As each particular word combination pat- tern describes a certain topic/concept in the document, the facts described above naturally lead to the hypothesis that each singular vector represents a salient topic/concept of the document, and the magnitude of its corresponding singular value represents the degree of importance of the salient topic/concept On the other hand, for query-relevant text summarization, the most common approach for performance evaluations, as showcased by the TIPSTER SUMMAC initiative 11], is that human evaluators use the automatically generated summary to judge the relevance of the original document to the user s query

Latent Semantic Indexing (LSI) CE-324: Modern Information Retrieval Sharif University of Technology

Latent Semantic Indexing (LSI) CE-324: Modern Information Retrieval Sharif University of Technology Latent Semantic Indexing (LSI) CE-324: Modern Information Retrieval Sharif University of Technology M. Soleymani Fall 2014 Most slides have been adapted from: Profs. Manning, Nayak & Raghavan (CS-276,

More information

Latent Semantic Indexing (LSI) CE-324: Modern Information Retrieval Sharif University of Technology

Latent Semantic Indexing (LSI) CE-324: Modern Information Retrieval Sharif University of Technology Latent Semantic Indexing (LSI) CE-324: Modern Information Retrieval Sharif University of Technology M. Soleymani Fall 2016 Most slides have been adapted from: Profs. Manning, Nayak & Raghavan (CS-276,

More information

Information Retrieval

Information Retrieval Introduction to Information CS276: Information and Web Search Christopher Manning and Pandu Nayak Lecture 13: Latent Semantic Indexing Ch. 18 Today s topic Latent Semantic Indexing Term-document matrices

More information

Lecture 5: Web Searching using the SVD

Lecture 5: Web Searching using the SVD Lecture 5: Web Searching using the SVD Information Retrieval Over the last 2 years the number of internet users has grown exponentially with time; see Figure. Trying to extract information from this exponentially

More information

CS47300: Web Information Search and Management

CS47300: Web Information Search and Management CS47300: Web Information Search and Management Prof. Chris Clifton 6 September 2017 Material adapted from course created by Dr. Luo Si, now leading Alibaba research group 1 Vector Space Model Disadvantages:

More information

Singular Value Decompsition

Singular Value Decompsition Singular Value Decompsition Massoud Malek One of the most useful results from linear algebra, is a matrix decomposition known as the singular value decomposition It has many useful applications in almost

More information

13 Searching the Web with the SVD

13 Searching the Web with the SVD 13 Searching the Web with the SVD 13.1 Information retrieval Over the last 20 years the number of internet users has grown exponentially with time; see Figure 1. Trying to extract information from this

More information

9 Searching the Internet with the SVD

9 Searching the Internet with the SVD 9 Searching the Internet with the SVD 9.1 Information retrieval Over the last 20 years the number of internet users has grown exponentially with time; see Figure 1. Trying to extract information from this

More information

Manning & Schuetze, FSNLP (c) 1999,2000

Manning & Schuetze, FSNLP (c) 1999,2000 558 15 Topics in Information Retrieval (15.10) y 4 3 2 1 0 0 1 2 3 4 5 6 7 8 Figure 15.7 An example of linear regression. The line y = 0.25x + 1 is the best least-squares fit for the four points (1,1),

More information

Manning & Schuetze, FSNLP, (c)

Manning & Schuetze, FSNLP, (c) page 554 554 15 Topics in Information Retrieval co-occurrence Latent Semantic Indexing Term 1 Term 2 Term 3 Term 4 Query user interface Document 1 user interface HCI interaction Document 2 HCI interaction

More information

1. Ignoring case, extract all unique words from the entire set of documents.

1. Ignoring case, extract all unique words from the entire set of documents. CS 378 Introduction to Data Mining Spring 29 Lecture 2 Lecturer: Inderjit Dhillon Date: Jan. 27th, 29 Keywords: Vector space model, Latent Semantic Indexing(LSI), SVD 1 Vector Space Model The basic idea

More information

Introduction to Information Retrieval

Introduction to Information Retrieval Introduction to Information Retrieval http://informationretrieval.org IIR 18: Latent Semantic Indexing Hinrich Schütze Center for Information and Language Processing, University of Munich 2013-07-10 1/43

More information

Latent Semantic Models. Reference: Introduction to Information Retrieval by C. Manning, P. Raghavan, H. Schutze

Latent Semantic Models. Reference: Introduction to Information Retrieval by C. Manning, P. Raghavan, H. Schutze Latent Semantic Models Reference: Introduction to Information Retrieval by C. Manning, P. Raghavan, H. Schutze 1 Vector Space Model: Pros Automatic selection of index terms Partial matching of queries

More information

Latent semantic indexing

Latent semantic indexing Latent semantic indexing Relationship between concepts and words is many-to-many. Solve problems of synonymy and ambiguity by representing documents as vectors of ideas or concepts, not terms. For retrieval,

More information

Latent Semantic Analysis. Hongning Wang

Latent Semantic Analysis. Hongning Wang Latent Semantic Analysis Hongning Wang CS@UVa Recap: vector space model Represent both doc and query by concept vectors Each concept defines one dimension K concepts define a high-dimensional space Element

More information

Natural Language Processing. Topics in Information Retrieval. Updated 5/10

Natural Language Processing. Topics in Information Retrieval. Updated 5/10 Natural Language Processing Topics in Information Retrieval Updated 5/10 Outline Introduction to IR Design features of IR systems Evaluation measures The vector space model Latent semantic indexing Background

More information

DATA MINING LECTURE 8. Dimensionality Reduction PCA -- SVD

DATA MINING LECTURE 8. Dimensionality Reduction PCA -- SVD DATA MINING LECTURE 8 Dimensionality Reduction PCA -- SVD The curse of dimensionality Real data usually have thousands, or millions of dimensions E.g., web documents, where the dimensionality is the vocabulary

More information

CSE 494/598 Lecture-6: Latent Semantic Indexing. **Content adapted from last year s slides

CSE 494/598 Lecture-6: Latent Semantic Indexing. **Content adapted from last year s slides CSE 494/598 Lecture-6: Latent Semantic Indexing LYDIA MANIKONDA HT TP://WWW.PUBLIC.ASU.EDU/~LMANIKON / **Content adapted from last year s slides Announcements Homework-1 and Quiz-1 Project part-2 released

More information

Linear Algebra Background

Linear Algebra Background CS76A Text Retrieval and Mining Lecture 5 Recap: Clustering Hierarchical clustering Agglomerative clustering techniques Evaluation Term vs. document space clustering Multi-lingual docs Feature selection

More information

Embeddings Learned By Matrix Factorization

Embeddings Learned By Matrix Factorization Embeddings Learned By Matrix Factorization Benjamin Roth; Folien von Hinrich Schütze Center for Information and Language Processing, LMU Munich Overview WordSpace limitations LinAlgebra review Input matrix

More information

Matrices, Vector Spaces, and Information Retrieval

Matrices, Vector Spaces, and Information Retrieval Matrices, Vector Spaces, and Information Authors: M. W. Berry and Z. Drmac and E. R. Jessup SIAM 1999: Society for Industrial and Applied Mathematics Speaker: Mattia Parigiani 1 Introduction Large volumes

More information

Variable Latent Semantic Indexing

Variable Latent Semantic Indexing Variable Latent Semantic Indexing Prabhakar Raghavan Yahoo! Research Sunnyvale, CA November 2005 Joint work with A. Dasgupta, R. Kumar, A. Tomkins. Yahoo! Research. Outline 1 Introduction 2 Background

More information

Boolean and Vector Space Retrieval Models

Boolean and Vector Space Retrieval Models Boolean and Vector Space Retrieval Models Many slides in this section are adapted from Prof. Joydeep Ghosh (UT ECE) who in turn adapted them from Prof. Dik Lee (Univ. of Science and Tech, Hong Kong) 1

More information

Dimension Reduction and Iterative Consensus Clustering

Dimension Reduction and Iterative Consensus Clustering Dimension Reduction and Iterative Consensus Clustering Southeastern Clustering and Ranking Workshop August 24, 2009 Dimension Reduction and Iterative 1 Document Clustering Geometry of the SVD Centered

More information

PV211: Introduction to Information Retrieval https://www.fi.muni.cz/~sojka/pv211

PV211: Introduction to Information Retrieval https://www.fi.muni.cz/~sojka/pv211 PV211: Introduction to Information Retrieval https://www.fi.muni.cz/~sojka/pv211 IIR 18: Latent Semantic Indexing Handout version Petr Sojka, Hinrich Schütze et al. Faculty of Informatics, Masaryk University,

More information

Latent Semantic Analysis. Hongning Wang

Latent Semantic Analysis. Hongning Wang Latent Semantic Analysis Hongning Wang CS@UVa VS model in practice Document and query are represented by term vectors Terms are not necessarily orthogonal to each other Synonymy: car v.s. automobile Polysemy:

More information

MODULE 8 Topics: Null space, range, column space, row space and rank of a matrix

MODULE 8 Topics: Null space, range, column space, row space and rank of a matrix MODULE 8 Topics: Null space, range, column space, row space and rank of a matrix Definition: Let L : V 1 V 2 be a linear operator. The null space N (L) of L is the subspace of V 1 defined by N (L) = {x

More information

Linear Algebra (Review) Volker Tresp 2017

Linear Algebra (Review) Volker Tresp 2017 Linear Algebra (Review) Volker Tresp 2017 1 Vectors k is a scalar (a number) c is a column vector. Thus in two dimensions, c = ( c1 c 2 ) (Advanced: More precisely, a vector is defined in a vector space.

More information

Introduction to Information Retrieval (Manning, Raghavan, Schutze) Chapter 6 Scoring term weighting and the vector space model

Introduction to Information Retrieval (Manning, Raghavan, Schutze) Chapter 6 Scoring term weighting and the vector space model Introduction to Information Retrieval (Manning, Raghavan, Schutze) Chapter 6 Scoring term weighting and the vector space model Ranked retrieval Thus far, our queries have all been Boolean. Documents either

More information

Information Retrieval and Topic Models. Mausam (Based on slides of W. Arms, Dan Jurafsky, Thomas Hofmann, Ata Kaban, Chris Manning, Melanie Martin)

Information Retrieval and Topic Models. Mausam (Based on slides of W. Arms, Dan Jurafsky, Thomas Hofmann, Ata Kaban, Chris Manning, Melanie Martin) Information Retrieval and Topic Models Mausam (Based on slides of W. Arms, Dan Jurafsky, Thomas Hofmann, Ata Kaban, Chris Manning, Melanie Martin) Sec. 1.1 Unstructured data in 1620 Which plays of Shakespeare

More information

DISTRIBUTIONAL SEMANTICS

DISTRIBUTIONAL SEMANTICS COMP90042 LECTURE 4 DISTRIBUTIONAL SEMANTICS LEXICAL DATABASES - PROBLEMS Manually constructed Expensive Human annotation can be biased and noisy Language is dynamic New words: slangs, terminology, etc.

More information

INF 141 IR METRICS LATENT SEMANTIC ANALYSIS AND INDEXING. Crista Lopes

INF 141 IR METRICS LATENT SEMANTIC ANALYSIS AND INDEXING. Crista Lopes INF 141 IR METRICS LATENT SEMANTIC ANALYSIS AND INDEXING Crista Lopes Outline Precision and Recall The problem with indexing so far Intuition for solving it Overview of the solution The Math How to measure

More information

Assignment 3. Latent Semantic Indexing

Assignment 3. Latent Semantic Indexing Assignment 3 Gagan Bansal 2003CS10162 Group 2 Pawan Jain 2003CS10177 Group 1 Latent Semantic Indexing OVERVIEW LATENT SEMANTIC INDEXING (LSI) considers documents that have many words in common to be semantically

More information

An Empirical Study on Dimensionality Optimization in Text Mining for Linguistic Knowledge Acquisition

An Empirical Study on Dimensionality Optimization in Text Mining for Linguistic Knowledge Acquisition An Empirical Study on Dimensionality Optimization in Text Mining for Linguistic Knowledge Acquisition Yu-Seop Kim 1, Jeong-Ho Chang 2, and Byoung-Tak Zhang 2 1 Division of Information and Telecommunication

More information

MATRICES. a m,1 a m,n A =

MATRICES. a m,1 a m,n A = MATRICES Matrices are rectangular arrays of real or complex numbers With them, we define arithmetic operations that are generalizations of those for real and complex numbers The general form a matrix of

More information

Retrieval by Content. Part 2: Text Retrieval Term Frequency and Inverse Document Frequency. Srihari: CSE 626 1

Retrieval by Content. Part 2: Text Retrieval Term Frequency and Inverse Document Frequency. Srihari: CSE 626 1 Retrieval by Content Part 2: Text Retrieval Term Frequency and Inverse Document Frequency Srihari: CSE 626 1 Text Retrieval Retrieval of text-based information is referred to as Information Retrieval (IR)

More information

Probabilistic Latent Semantic Analysis

Probabilistic Latent Semantic Analysis Probabilistic Latent Semantic Analysis Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr

More information

Quick Introduction to Nonnegative Matrix Factorization

Quick Introduction to Nonnegative Matrix Factorization Quick Introduction to Nonnegative Matrix Factorization Norm Matloff University of California at Davis 1 The Goal Given an u v matrix A with nonnegative elements, we wish to find nonnegative, rank-k matrices

More information

Leverage Sparse Information in Predictive Modeling

Leverage Sparse Information in Predictive Modeling Leverage Sparse Information in Predictive Modeling Liang Xie Countrywide Home Loans, Countrywide Bank, FSB August 29, 2008 Abstract This paper examines an innovative method to leverage information from

More information

Notes on Latent Semantic Analysis

Notes on Latent Semantic Analysis Notes on Latent Semantic Analysis Costas Boulis 1 Introduction One of the most fundamental problems of information retrieval (IR) is to find all documents (and nothing but those) that are semantically

More information

INFO 4300 / CS4300 Information Retrieval. IR 9: Linear Algebra Review

INFO 4300 / CS4300 Information Retrieval. IR 9: Linear Algebra Review INFO 4300 / CS4300 Information Retrieval IR 9: Linear Algebra Review Paul Ginsparg Cornell University, Ithaca, NY 24 Sep 2009 1/ 23 Overview 1 Recap 2 Matrix basics 3 Matrix Decompositions 4 Discussion

More information

A Note on the Effect of Term Weighting on Selecting Intrinsic Dimensionality of Data

A Note on the Effect of Term Weighting on Selecting Intrinsic Dimensionality of Data BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 9, No 1 Sofia 2009 A Note on the Effect of Term Weighting on Selecting Intrinsic Dimensionality of Data Ch. Aswani Kumar 1,

More information

COMS 4721: Machine Learning for Data Science Lecture 18, 4/4/2017

COMS 4721: Machine Learning for Data Science Lecture 18, 4/4/2017 COMS 4721: Machine Learning for Data Science Lecture 18, 4/4/2017 Prof. John Paisley Department of Electrical Engineering & Data Science Institute Columbia University TOPIC MODELING MODELS FOR TEXT DATA

More information

Linear Systems. Carlo Tomasi

Linear Systems. Carlo Tomasi Linear Systems Carlo Tomasi Section 1 characterizes the existence and multiplicity of the solutions of a linear system in terms of the four fundamental spaces associated with the system s matrix and of

More information

Text Analytics (Text Mining)

Text Analytics (Text Mining) http://poloclub.gatech.edu/cse6242 CSE6242 / CX4242: Data & Visual Analytics Text Analytics (Text Mining) Concepts, Algorithms, LSI/SVD Duen Horng (Polo) Chau Assistant Professor Associate Director, MS

More information

CS 3750 Advanced Machine Learning. Applications of SVD and PCA (LSA and Link analysis) Cem Akkaya

CS 3750 Advanced Machine Learning. Applications of SVD and PCA (LSA and Link analysis) Cem Akkaya CS 375 Advanced Machine Learning Applications of SVD and PCA (LSA and Link analysis) Cem Akkaya Outline SVD and LSI Kleinberg s Algorithm PageRank Algorithm Vector Space Model Vector space model represents

More information

Fall CS646: Information Retrieval. Lecture 6 Boolean Search and Vector Space Model. Jiepu Jiang University of Massachusetts Amherst 2016/09/26

Fall CS646: Information Retrieval. Lecture 6 Boolean Search and Vector Space Model. Jiepu Jiang University of Massachusetts Amherst 2016/09/26 Fall 2016 CS646: Information Retrieval Lecture 6 Boolean Search and Vector Space Model Jiepu Jiang University of Massachusetts Amherst 2016/09/26 Outline Today Boolean Retrieval Vector Space Model Latent

More information

Data Mining and Matrices

Data Mining and Matrices Data Mining and Matrices 05 Semi-Discrete Decomposition Rainer Gemulla, Pauli Miettinen May 16, 2013 Outline 1 Hunting the Bump 2 Semi-Discrete Decomposition 3 The Algorithm 4 Applications SDD alone SVD

More information

USING SINGULAR VALUE DECOMPOSITION (SVD) AS A SOLUTION FOR SEARCH RESULT CLUSTERING

USING SINGULAR VALUE DECOMPOSITION (SVD) AS A SOLUTION FOR SEARCH RESULT CLUSTERING POZNAN UNIVE RSIY OF E CHNOLOGY ACADE MIC JOURNALS No. 80 Electrical Engineering 2014 Hussam D. ABDULLA* Abdella S. ABDELRAHMAN* Vaclav SNASEL* USING SINGULAR VALUE DECOMPOSIION (SVD) AS A SOLUION FOR

More information

Matrix decompositions and latent semantic indexing

Matrix decompositions and latent semantic indexing 18 Matrix decompositions and latent semantic indexing On page 113, we introduced the notion of a term-document matrix: an M N matrix C, each of whose rows represents a term and each of whose columns represents

More information

The value of a problem is not so much coming up with the answer as in the ideas and attempted ideas it forces on the would be solver I.N.

The value of a problem is not so much coming up with the answer as in the ideas and attempted ideas it forces on the would be solver I.N. Math 410 Homework Problems In the following pages you will find all of the homework problems for the semester. Homework should be written out neatly and stapled and turned in at the beginning of class

More information

Comparative Summarization via Latent Dirichlet Allocation

Comparative Summarization via Latent Dirichlet Allocation Comparative Summarization via Latent Dirichlet Allocation Michal Campr and Karel Jezek Department of Computer Science and Engineering, FAV, University of West Bohemia, 11 February 2013, 301 00, Plzen,

More information

Chapter 6. Orthogonality

Chapter 6. Orthogonality 6.4 The Projection Matrix 1 Chapter 6. Orthogonality 6.4 The Projection Matrix Note. In Section 6.1 (Projections), we projected a vector b R n onto a subspace W of R n. We did so by finding a basis for

More information

Structured matrix factorizations. Example: Eigenfaces

Structured matrix factorizations. Example: Eigenfaces Structured matrix factorizations Example: Eigenfaces An extremely large variety of interesting and important problems in machine learning can be formulated as: Given a matrix, find a matrix and a matrix

More information

Linear Algebra, Summer 2011, pt. 3

Linear Algebra, Summer 2011, pt. 3 Linear Algebra, Summer 011, pt. 3 September 0, 011 Contents 1 Orthogonality. 1 1.1 The length of a vector....................... 1. Orthogonal vectors......................... 3 1.3 Orthogonal Subspaces.......................

More information

Information Retrieval and Web Search

Information Retrieval and Web Search Information Retrieval and Web Search IR models: Vector Space Model IR Models Set Theoretic Classic Models Fuzzy Extended Boolean U s e r T a s k Retrieval: Adhoc Filtering Brosing boolean vector probabilistic

More information

Vector Space Models. wine_spectral.r

Vector Space Models. wine_spectral.r Vector Space Models 137 wine_spectral.r Latent Semantic Analysis Problem with words Even a small vocabulary as in wine example is challenging LSA Reduce number of columns of DTM by principal components

More information

Linear Algebra (Review) Volker Tresp 2018

Linear Algebra (Review) Volker Tresp 2018 Linear Algebra (Review) Volker Tresp 2018 1 Vectors k, M, N are scalars A one-dimensional array c is a column vector. Thus in two dimensions, ( ) c1 c = c 2 c i is the i-th component of c c T = (c 1, c

More information

RETRIEVAL MODELS. Dr. Gjergji Kasneci Introduction to Information Retrieval WS

RETRIEVAL MODELS. Dr. Gjergji Kasneci Introduction to Information Retrieval WS RETRIEVAL MODELS Dr. Gjergji Kasneci Introduction to Information Retrieval WS 2012-13 1 Outline Intro Basics of probability and information theory Retrieval models Boolean model Vector space model Probabilistic

More information

Linear Algebra I Lecture 8

Linear Algebra I Lecture 8 Linear Algebra I Lecture 8 Xi Chen 1 1 University of Alberta January 25, 2019 Outline 1 2 Gauss-Jordan Elimination Given a system of linear equations f 1 (x 1, x 2,..., x n ) = 0 f 2 (x 1, x 2,..., x n

More information

CS276A Text Information Retrieval, Mining, and Exploitation. Lecture 4 15 Oct 2002

CS276A Text Information Retrieval, Mining, and Exploitation. Lecture 4 15 Oct 2002 CS276A Text Information Retrieval, Mining, and Exploitation Lecture 4 15 Oct 2002 Recap of last time Index size Index construction techniques Dynamic indices Real world considerations 2 Back of the envelope

More information

Updating the Centroid Decomposition with Applications in LSI

Updating the Centroid Decomposition with Applications in LSI Updating the Centroid Decomposition with Applications in LSI Jason R. Blevins and Moody T. Chu Department of Mathematics, N.C. State University May 14, 24 Abstract The centroid decomposition (CD) is an

More information

CS 572: Information Retrieval

CS 572: Information Retrieval CS 572: Information Retrieval Lecture 11: Topic Models Acknowledgments: Some slides were adapted from Chris Manning, and from Thomas Hoffman 1 Plan for next few weeks Project 1: done (submit by Friday).

More information

Machine Learning. Principal Components Analysis. Le Song. CSE6740/CS7641/ISYE6740, Fall 2012

Machine Learning. Principal Components Analysis. Le Song. CSE6740/CS7641/ISYE6740, Fall 2012 Machine Learning CSE6740/CS7641/ISYE6740, Fall 2012 Principal Components Analysis Le Song Lecture 22, Nov 13, 2012 Based on slides from Eric Xing, CMU Reading: Chap 12.1, CB book 1 2 Factor or Component

More information

.. CSC 566 Advanced Data Mining Alexander Dekhtyar..

.. CSC 566 Advanced Data Mining Alexander Dekhtyar.. .. CSC 566 Advanced Data Mining Alexander Dekhtyar.. Information Retrieval Latent Semantic Indexing Preliminaries Vector Space Representation of Documents: TF-IDF Documents. A single text document is a

More information

Sparse vectors recap. ANLP Lecture 22 Lexical Semantics with Dense Vectors. Before density, another approach to normalisation.

Sparse vectors recap. ANLP Lecture 22 Lexical Semantics with Dense Vectors. Before density, another approach to normalisation. ANLP Lecture 22 Lexical Semantics with Dense Vectors Henry S. Thompson Based on slides by Jurafsky & Martin, some via Dorota Glowacka 5 November 2018 Previous lectures: Sparse vectors recap How to represent

More information

ANLP Lecture 22 Lexical Semantics with Dense Vectors

ANLP Lecture 22 Lexical Semantics with Dense Vectors ANLP Lecture 22 Lexical Semantics with Dense Vectors Henry S. Thompson Based on slides by Jurafsky & Martin, some via Dorota Glowacka 5 November 2018 Henry S. Thompson ANLP Lecture 22 5 November 2018 Previous

More information

Linear Systems. Carlo Tomasi. June 12, r = rank(a) b range(a) n r solutions

Linear Systems. Carlo Tomasi. June 12, r = rank(a) b range(a) n r solutions Linear Systems Carlo Tomasi June, 08 Section characterizes the existence and multiplicity of the solutions of a linear system in terms of the four fundamental spaces associated with the system s matrix

More information

22m:033 Notes: 3.1 Introduction to Determinants

22m:033 Notes: 3.1 Introduction to Determinants 22m:033 Notes: 3. Introduction to Determinants Dennis Roseman University of Iowa Iowa City, IA http://www.math.uiowa.edu/ roseman October 27, 2009 When does a 2 2 matrix have an inverse? ( ) a a If A =

More information

CS264: Beyond Worst-Case Analysis Lecture #15: Topic Modeling and Nonnegative Matrix Factorization

CS264: Beyond Worst-Case Analysis Lecture #15: Topic Modeling and Nonnegative Matrix Factorization CS264: Beyond Worst-Case Analysis Lecture #15: Topic Modeling and Nonnegative Matrix Factorization Tim Roughgarden February 28, 2017 1 Preamble This lecture fulfills a promise made back in Lecture #1,

More information

Lecture: Face Recognition and Feature Reduction

Lecture: Face Recognition and Feature Reduction Lecture: Face Recognition and Feature Reduction Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab 1 Recap - Curse of dimensionality Assume 5000 points uniformly distributed in the

More information

Linear Algebra Methods for Data Mining

Linear Algebra Methods for Data Mining Linear Algebra Methods for Data Mining Saara Hyvönen, Saara.Hyvonen@cs.helsinki.fi Spring 2007 1. Basic Linear Algebra Linear Algebra Methods for Data Mining, Spring 2007, University of Helsinki Example

More information

Fast LSI-based techniques for query expansion in text retrieval systems

Fast LSI-based techniques for query expansion in text retrieval systems Fast LSI-based techniques for query expansion in text retrieval systems L. Laura U. Nanni F. Sarracco Department of Computer and System Science University of Rome La Sapienza 2nd Workshop on Text-based

More information

Matrix Decomposition and Latent Semantic Indexing (LSI) Introduction to Information Retrieval INF 141/ CS 121 Donald J. Patterson

Matrix Decomposition and Latent Semantic Indexing (LSI) Introduction to Information Retrieval INF 141/ CS 121 Donald J. Patterson Matrix Decomposition and Latent Semantic Indexing (LSI) Introduction to Information Retrieval INF 141/ CS 121 Donald J. Patterson Latent Semantic Indexing Outline Introduction Linear Algebra Refresher

More information

Semantics with Dense Vectors. Reference: D. Jurafsky and J. Martin, Speech and Language Processing

Semantics with Dense Vectors. Reference: D. Jurafsky and J. Martin, Speech and Language Processing Semantics with Dense Vectors Reference: D. Jurafsky and J. Martin, Speech and Language Processing 1 Semantics with Dense Vectors We saw how to represent a word as a sparse vector with dimensions corresponding

More information

(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =

(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax = . (5 points) (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? dim N(A), since rank(a) 3. (b) If we also know that Ax = has no solution, what do we know about the rank of A? C(A)

More information

Information Retrieval

Information Retrieval Introduction to Information Retrieval CS276: Information Retrieval and Web Search Christopher Manning and Prabhakar Raghavan Lecture 6: Scoring, Term Weighting and the Vector Space Model This lecture;

More information

Chapter 3: Theory Review: Solutions Math 308 F Spring 2015

Chapter 3: Theory Review: Solutions Math 308 F Spring 2015 Chapter : Theory Review: Solutions Math 08 F Spring 05. What two properties must a function T : R m R n satisfy to be a linear transformation? (a) For all vectors u and v in R m, T (u + v) T (u) + T (v)

More information

AM 205: lecture 8. Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition

AM 205: lecture 8. Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition AM 205: lecture 8 Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition QR Factorization A matrix A R m n, m n, can be factorized

More information

Image Registration Lecture 2: Vectors and Matrices

Image Registration Lecture 2: Vectors and Matrices Image Registration Lecture 2: Vectors and Matrices Prof. Charlene Tsai Lecture Overview Vectors Matrices Basics Orthogonal matrices Singular Value Decomposition (SVD) 2 1 Preliminary Comments Some of this

More information

Linear Algebra (part 1) : Vector Spaces (by Evan Dummit, 2017, v. 1.07) 1.1 The Formal Denition of a Vector Space

Linear Algebra (part 1) : Vector Spaces (by Evan Dummit, 2017, v. 1.07) 1.1 The Formal Denition of a Vector Space Linear Algebra (part 1) : Vector Spaces (by Evan Dummit, 2017, v. 1.07) Contents 1 Vector Spaces 1 1.1 The Formal Denition of a Vector Space.................................. 1 1.2 Subspaces...................................................

More information

Vector Spaces, Orthogonality, and Linear Least Squares

Vector Spaces, Orthogonality, and Linear Least Squares Week Vector Spaces, Orthogonality, and Linear Least Squares. Opening Remarks.. Visualizing Planes, Lines, and Solutions Consider the following system of linear equations from the opener for Week 9: χ χ

More information

INFO 4300 / CS4300 Information Retrieval. slides adapted from Hinrich Schütze s, linked from

INFO 4300 / CS4300 Information Retrieval. slides adapted from Hinrich Schütze s, linked from INFO 4300 / CS4300 Information Retrieval slides adapted from Hinrich Schütze s, linked from http://informationretrieval.org/ IR 8: Evaluation & SVD Paul Ginsparg Cornell University, Ithaca, NY 20 Sep 2011

More information

Lecture: Face Recognition and Feature Reduction

Lecture: Face Recognition and Feature Reduction Lecture: Face Recognition and Feature Reduction Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab Lecture 11-1 Recap - Curse of dimensionality Assume 5000 points uniformly distributed

More information

CSE 494/598 Lecture-4: Correlation Analysis. **Content adapted from last year s slides

CSE 494/598 Lecture-4: Correlation Analysis. **Content adapted from last year s slides CSE 494/598 Lecture-4: Correlation Analysis LYDIA MANIKONDA HT TP://WWW.PUBLIC.ASU.EDU/~LMANIKON / **Content adapted from last year s slides Announcements Project-1 Due: February 12 th 2016 Analysis report:

More information

Accuracy of Distributed Data Retrieval on a Supercomputer Grid. Technical Report December Department of Scientific Computing

Accuracy of Distributed Data Retrieval on a Supercomputer Grid. Technical Report December Department of Scientific Computing Accuracy of Distributed Data Retrieval on a Supercomputer Grid Gabriel Okša a Marián Vajteršic a Institute of Mathematics, Department of Informatics, Slovak Academy of Sciences, Bratislava, Slovak Republic

More information

EIGENVALE PROBLEMS AND THE SVD. [5.1 TO 5.3 & 7.4]

EIGENVALE PROBLEMS AND THE SVD. [5.1 TO 5.3 & 7.4] EIGENVALE PROBLEMS AND THE SVD. [5.1 TO 5.3 & 7.4] Eigenvalue Problems. Introduction Let A an n n real nonsymmetric matrix. The eigenvalue problem: Au = λu λ C : eigenvalue u C n : eigenvector Example:

More information

Applied Mathematics 205. Unit II: Numerical Linear Algebra. Lecturer: Dr. David Knezevic

Applied Mathematics 205. Unit II: Numerical Linear Algebra. Lecturer: Dr. David Knezevic Applied Mathematics 205 Unit II: Numerical Linear Algebra Lecturer: Dr. David Knezevic Unit II: Numerical Linear Algebra Chapter II.3: QR Factorization, SVD 2 / 66 QR Factorization 3 / 66 QR Factorization

More information

Linear Algebra Methods for Data Mining

Linear Algebra Methods for Data Mining Linear Algebra Methods for Data Mining Saara Hyvönen, Saara.Hyvonen@cs.helsinki.fi Spring 2007 The Singular Value Decomposition (SVD) continued Linear Algebra Methods for Data Mining, Spring 2007, University

More information

Matrix Factorization & Latent Semantic Analysis Review. Yize Li, Lanbo Zhang

Matrix Factorization & Latent Semantic Analysis Review. Yize Li, Lanbo Zhang Matrix Factorization & Latent Semantic Analysis Review Yize Li, Lanbo Zhang Overview SVD in Latent Semantic Indexing Non-negative Matrix Factorization Probabilistic Latent Semantic Indexing Vector Space

More information

Matrices, Vector Spaces, and Information Retrieval

Matrices, Vector Spaces, and Information Retrieval SIAM REVIEW Vol. 41, No. 2, pp. 335 362 c 1999 Society for Industrial and Applied Mathematics Matrices, Vector Spaces, and Information Retrieval Michael W. Berry Zlatko Drmač Elizabeth R. Jessup Abstract.

More information

Scoring, Term Weighting and the Vector Space

Scoring, Term Weighting and the Vector Space Scoring, Term Weighting and the Vector Space Model Francesco Ricci Most of these slides comes from the course: Information Retrieval and Web Search, Christopher Manning and Prabhakar Raghavan Content [J

More information

Information Retrieval Basic IR models. Luca Bondi

Information Retrieval Basic IR models. Luca Bondi Basic IR models Luca Bondi Previously on IR 2 d j q i IRM SC q i, d j IRM D, Q, R q i, d j d j = w 1,j, w 2,j,, w M,j T w i,j = 0 if term t i does not appear in document d j w i,j and w i:1,j assumed to

More information

How Latent Semantic Indexing Solves the Pachyderm Problem

How Latent Semantic Indexing Solves the Pachyderm Problem How Latent Semantic Indexing Solves the Pachyderm Problem Michael A. Covington Institute for Artificial Intelligence The University of Georgia 2011 1 Introduction Here I present a brief mathematical demonstration

More information

MAT2342 : Introduction to Applied Linear Algebra Mike Newman, fall Projections. introduction

MAT2342 : Introduction to Applied Linear Algebra Mike Newman, fall Projections. introduction MAT4 : Introduction to Applied Linear Algebra Mike Newman fall 7 9. Projections introduction One reason to consider projections is to understand approximate solutions to linear systems. A common example

More information

SVD, Power method, and Planted Graph problems (+ eigenvalues of random matrices)

SVD, Power method, and Planted Graph problems (+ eigenvalues of random matrices) Chapter 14 SVD, Power method, and Planted Graph problems (+ eigenvalues of random matrices) Today we continue the topic of low-dimensional approximation to datasets and matrices. Last time we saw the singular

More information

Introduction to Data Mining

Introduction to Data Mining Introduction to Data Mining Lecture #21: Dimensionality Reduction Seoul National University 1 In This Lecture Understand the motivation and applications of dimensionality reduction Learn the definition

More information

Jeffrey D. Ullman Stanford University

Jeffrey D. Ullman Stanford University Jeffrey D. Ullman Stanford University 2 Often, our data can be represented by an m-by-n matrix. And this matrix can be closely approximated by the product of two matrices that share a small common dimension

More information

Scoring (Vector Space Model) CE-324: Modern Information Retrieval Sharif University of Technology

Scoring (Vector Space Model) CE-324: Modern Information Retrieval Sharif University of Technology Scoring (Vector Space Model) CE-324: Modern Information Retrieval Sharif University of Technology M. Soleymani Fall 2014 Most slides have been adapted from: Profs. Manning, Nayak & Raghavan (CS-276, Stanford)

More information