Text Summarisation for Evidence Based Medicine
|
|
|
- Julius Wilson
- 6 years ago
- Views:
Transcription
1 Text Summarisation for Evidence Based Medicine Diego Mollá Centre for Language Technology, Macquarie University IIT Patna, 16 December 2012
2 Contents Evidence Based Medicine What is Evidence Based Medicine? EBM and NLP A Corpus for Summarisation Text Summarisation Sentence Extraction Cohesion Check Balance and Coverage Proposals for Text Summarisation Single-document Summarisation Optimisation and Summarisation EBM Summarisation Diego Mollá 2/79
3 Evidence Based Medicine Text Summarisation Proposals for Text Summarisation About us: Macquarie University EBM Summarisation Diego Molla 3/79
4 About us: Centre for Language Technology Core Staff (* involved in the AISRF project) Prof. Robert Dale * Prof. Mark Johnson * A. Prof. Mark Dras A. Prof. Steve Cassidy * Dr. Diego Molla-Aliod Dr. Rolf Schwitter EBM Summarisation Diego Mollá 4/79
5 About Us: Research Group on Natural Language Processing of Medical Texts Active Members Diego Mollá Senior lecturer at Macquarie University. Abeed Sarker PhD student at Macquarie University. Sara Faisal Shash Masters student. Past Members María Elena Santiago-Martínez Research programmer. Patrick Davis-Desmond Masters student. Andreea Tutos Masters student. EBM Summarisation Diego Mollá 5/79
6 About Me: Diego Mollá-Aliod Some Highlights MSc (1992), PhD (1996) University of Edinburgh. ExtrAns and WebExtrAns projects at University of Zurich. AnswerFinder project and Medical NLP research at Macquarie University. Research interests Question Answering. Summarisation. Information Extraction. EBM Summarisation Diego Mollá 6/79
7 Contents Evidence Based Medicine What is Evidence Based Medicine? EBM and NLP A Corpus for Summarisation Text Summarisation Sentence Extraction Cohesion Check Balance and Coverage Proposals for Text Summarisation Single-document Summarisation Optimisation and Summarisation EBM Summarisation Diego Mollá 7/79
8 Contents Evidence Based Medicine What is Evidence Based Medicine? EBM and NLP A Corpus for Summarisation Text Summarisation Sentence Extraction Cohesion Check Balance and Coverage Proposals for Text Summarisation Single-document Summarisation Optimisation and Summarisation EBM Summarisation Diego Mollá 8/79
9 Evidence Based Medicine EBM Summarisation Diego Mollá 9/79
10 Suggested Steps in EBM EBM Summarisation Diego Mollá 10/79
11 PICO for Asking the Right Question EBM Summarisation Diego Mollá 11/79
12 Where to search for external evidence? 1. Evidence-based Summaries (Systematic Reviews): The Cochrane Library ( EBM Online ( UptoDate ( EBM Summarisation Diego Mollá 12/79
13 Where to search for external evidence? 1. Evidence-based Summaries (Systematic Reviews): The Cochrane Library ( EBM Online ( UptoDate ( 2. Search the Medical Literature: E.g. PubMed ( EBM Summarisation Diego Mollá 12/79
14 Searching Cochrane EBM Summarisation Diego Mollá 13/79
15 Searching PubMed EBM Summarisation Diego Mollá 14/79
16 Searching the Trip Database EBM Summarisation Diego Mollá 15/79
17 Appraising the Evidence The SORT Taxonomy Level A Consistent and good-quality patient-oriented evidence. Level B Inconsistent or limited-quality patient-oriented evidence. Level C Consensus, usual practise, opinion, disease-oriented evidence, or case series for studies of diagnosis, treatment, prevention, or screening. EBM Summarisation Diego Mollá 16/79
18 Contents Evidence Based Medicine What is Evidence Based Medicine? EBM and NLP A Corpus for Summarisation Text Summarisation Sentence Extraction Cohesion Check Balance and Coverage Proposals for Text Summarisation Single-document Summarisation Optimisation and Summarisation EBM Summarisation Diego Mollá 17/79
19 Where can NLP Help? Questions: Help formulate answerable questions. From natural question to PICO frames? Question analysis and classification. EBM Summarisation Diego Mollá 18/79
20 Where can NLP Help? Questions: Help formulate answerable questions. From natural question to PICO frames? Question analysis and classification. Search: Retrieve and rank relevant literature. Extract the evidence-based information. Summarise the results. EBM Summarisation Diego Mollá 18/79
21 Where can NLP Help? (II) Appraisal: Classify the evidence. EBM Summarisation Diego Mollá 19/79
22 Contents Evidence Based Medicine What is Evidence Based Medicine? EBM and NLP A Corpus for Summarisation Text Summarisation Sentence Extraction Cohesion Check Balance and Coverage Proposals for Text Summarisation Single-document Summarisation Optimisation and Summarisation EBM Summarisation Diego Mollá 20/79
23 Where s the Corpus for Summarisation? Summarisation Systems CENTRIFUSER/PERSIVAL: Developed and tested using user feedback (iterative design). SemRep: Evaluation based on human judgement. Demner-Fushman & Lin: ROUGE on original paper abstracts. Fiszman: Factoid-based evaluation. EBM Summarisation Diego Mollá 21/79
24 Where s the Corpus for Summarisation? Summarisation Systems CENTRIFUSER/PERSIVAL: Developed and tested using user feedback (iterative design). SemRep: Evaluation based on human judgement. Demner-Fushman & Lin: ROUGE on original paper abstracts. Fiszman: Factoid-based evaluation. Corpora Several corpora of questions/answers available. Answers lack explicit pointers to primary literature. Medical doctors want to know the primary sources. EBM Summarisation Diego Mollá 21/79
25 Journal of Family Practice s Clinical Inquiries EBM Summarisation Diego Mollá 22/79
26 The XML Contents I <u r l >h t t p : / /www. j f p o n l i n e. com/ Pages. asp?aid=7843& ; i s s u e=september 2009& ; UID=</u r l > <record id = 7843 > <question>which treatments work best f o r hemorrhoids?</question> <answer> <s n i p i d = 1 > <s n i p t e x t >E x c i s i o n i s t h e most e f f e c t i v e t r e a t m e n t f o r thrombosed e x t e r n a l h e m o r r h o i d s.</ s n i p t e x t > <s o r t y p e= B > r e t r o s p e c t i v e s t u d i e s </sor> <l o n g i d = 1 1 > <l o n g t e x t >A r e t r o s p e c t i v e s t u d y o f 231 p a t i e n t s t r e a t e d c o n s e r v a t i v e l y o r s u r g i c a l l y found t h a t t h e 48.5% o f p a t i e n t s t r e a t e d s u r g i c a l l y had a l o w e r r e c u r r e n c e r a t e than t h e c o n s e r v a t i v e group ( number needed to t r e a t [NNT]=2 f o r recurrence at mean follow up of 7. 6 months ) and e a r l i e r r e s o l u t i o n o f symptoms ( a v e r a g e 3. 9 days compared w i t h 24 days f o r c o n s e r v a t i v e t r e a t m e n t ).</ l o n g t e x t > <r e f i d = a b s t r a c t = A b s t r a c t s / xml >Greenspon J, W i l l i a m s SB, Young HA, e t a l. Thrombosed e x t e r n a l h e m o r r h o i d s : outcome a f t e r c o n s e r v a t i v e o r s u r g i c a l management. Dis Colon Rectum. 2004; 47: </ ref> </long> <l o n g i d = 1 2 > <l o n g t e x t >A r e t r o s p e c t i v e a n a l y s i s o f 340 p a t i e n t s who underwent o u t p a t i e n t e x c i s i o n o f thrombosed e x t e r n a l h e m o r r h o i d s under l o c a l a n e s t h e s i a r e p o r t e d a low r e c u r r e n c e r a t e o f 6.5% a t a EBM Summarisation Diego Mollá 23/79
27 The XML Contents II mean f o l l o w up o f months.</ l o n g t e x t > <r e f i d = a b s t r a c t = A b s t r a c t s / xml >Jongen J, Bach S, S t u b i n g e r SH, e t a l. E x c i s i o n o f thrombosed e x t e r n a l h e m o r r h o i d s under l o c a l a n e s t h e s i a : a r e t r o s p e c t i v e e v a l u a t i o n of 340 p a t i e n t s. Dis Colon Rectum. 2003; 46: </ ref> </long> <l o n g i d = 1 3 > <l o n g t e x t >A p r o s p e c t i v e, randomized c o n t r o l l e d t r i a l (RCT) o f 98 p a t i e n t s t r e a t e d n o n s u r g i c a l l y found improved p a i n r e l i e f w i t h a c o m b i n a t i o n o f t o p i c a l n i f e d i p i n e 0.3% and l i d o c a i n e 1.5% compared w i t h l i d o c a i n e a l o n e. The NNT f o r complete p a i n r e l i e f a t 7 days was 3.</ l o n g t e x t > <r e f i d = a b s t r a c t = A b s t r a c t s / xml > P e r r o t t i P, Antropoli C, Molino D, et a l. Conservative treatment of acute thrombosed e x t e r n a l h e m o r r h o i d s w i t h t o p i c a l n i f e d i p i n e. Dis Colon Rectum. 2001; 44: </ ref> </long> </s n i p> </answer> </r e c o r d> EBM Summarisation Diego Mollá 24/79
28 Components of the Corpus Question Direct extract from the source. Answer Split from the source and manually checked. Evidence Extracted from the source. Additional text Manually extracted from the source and massaged. References PMID looked up in PubMed (automatic and manual procedure). EBM Summarisation Diego Mollá 25/79
29 Corpus Statistics Size 456 questions ( records ). 1,396 answer parts ( snips ). 3,036 answer justifications ( longs ). 3,705 references: 2,908 unique references. 2,657 XML abstracts from PubMed. EBM Summarisation Diego Mollá 26/79
30 Answer parts per Question Avg=3.06 EBM Summarisation Diego Mollá 27/79
31 Answer justifications per answer part Avg=2.17 EBM Summarisation Diego Mollá 28/79
32 References per answer justification Avg=1.22 EBM Summarisation Diego Mollá 29/79
33 References per question Avg=6.57 EBM Summarisation Diego Mollá 30/79
34 Evidence Grade EBM Summarisation Diego Mollá 31/79
35 References EBM Summarisation Diego Mollá 32/79
36 Contents Evidence Based Medicine What is Evidence Based Medicine? EBM and NLP A Corpus for Summarisation Text Summarisation Sentence Extraction Cohesion Check Balance and Coverage Proposals for Text Summarisation Single-document Summarisation Optimisation and Summarisation EBM Summarisation Diego Mollá 33/79
37 What is Document Summarisation Summarisation (or automatic abstracting) A summary is a text that is produced from one or more texts, that contains a significant portion of the information of the original text(s), and that is no longer than half of the original text(s). (Hovy, 2003) EBM Summarisation Diego Mollá 34/79
38 What is Document Summarisation Good For? What for? For busy people to read the summary instead of the full text. EBM Summarisation Diego Mollá 35/79
39 What is Document Summarisation Good For? What for? For busy people to read the summary instead of the full text. informative summary EBM Summarisation Diego Mollá 35/79
40 What is Document Summarisation Good For? What for? For busy people to read the summary instead of the full text. informative summary For researchers, web surfers,... to read the summary to decide if it is worth to read the original text. EBM Summarisation Diego Mollá 35/79
41 What is Document Summarisation Good For? What for? For busy people to read the summary instead of the full text. informative summary For researchers, web surfers,... to read the summary to decide if it is worth to read the original text. indicative summary EBM Summarisation Diego Mollá 35/79
42 What is Document Summarisation Good For? What for? For busy people to read the summary instead of the full text. informative summary For researchers, web surfers,... to read the summary to decide if it is worth to read the original text. indicative summary To avoid having to type out an abstract for a technical report when the publisher requests it. EBM Summarisation Diego Mollá 35/79
43 What is Document Summarisation Good For? What for? For busy people to read the summary instead of the full text. informative summary For researchers, web surfers,... to read the summary to decide if it is worth to read the original text. indicative summary To avoid having to type out an abstract for a technical report when the publisher requests it. not realistic EBM Summarisation Diego Mollá 35/79
44 Examples I Original Text Abraham Lincoln was born in Kentucky on February 12th, His family moved to Indiana when he was eight years old. His mother died when he was ten. Abraham Lincoln loved to read. He only went to school for a few months. He had to study and learn on his own. From his childhood, Abraham Lincoln was known as a hard worker. He worked on a farm and in a store and on a boat. He studied law and practiced for many years in Illinois. He married Mary Todd Lincoln. Together they had four sons. Abraham Lincoln was elected as the 16th President of the United States in He did many things as President. Many people think he was the best American President of all time. He is most remembered for freeing the slaves. He was President of the United States during the time the Civil War was fought. The Civil War was fought between the Northern and Southern states. He was known as a great speaker. Some of his most famous speeches include the Emancipation Proclamation - in which he freed the slaves - and the Gettysburg Address, which starts Four score and seven years ago... While attending a play on April 14th, 1865, Abraham Lincoln was shot by John Wilkes Booth. Lincoln died the next morning. Lincoln s birthday is celebrated in February of each year. EBM Summarisation Diego Mollá 36/79
45 Examples II Word s Autosummarize Abraham Lincoln was born in Kentucky on February 12th, Abraham Lincoln loved to read. Abraham Lincoln was elected as the 16th President of the United States in The Civil War was fought between the Northern and Southern states. While attending a play on April 14th, 1865, Abraham Lincoln was shot by John Wilkes Booth. Lincoln died the next morning. MEAD Abraham Lincoln was born in Kentucky on February 12th His family moved to Indiana when he was eight years old. From his childhood Abraham Lincoln was known as a hard worker. Abraham Lincoln was elected as the 16th President of the United States in While attending a play on April 14th 1865 Abraham Lincoln was shot by John Wilkes Booth. EBM Summarisation Diego Mollá 37/79
46 An Ideal Document Summarisation System Understanding Stage Document(s) Knowledge base EBM Summarisation Diego Mollá 38/79
47 An Ideal Document Summarisation System Understanding Stage Document(s) Knowledge base Generation Stage Knowledge base Summary EBM Summarisation Diego Mollá 38/79
48 A Compromise Solution Sentence Extraction Document Sentence candidates EBM Summarisation Diego Mollá 39/79
49 A Compromise Solution Sentence Extraction Document Sentence candidates Cohesion Check Sentence candidates Coherent text EBM Summarisation Diego Mollá 39/79
50 A Compromise Solution Sentence Extraction Document Sentence candidates Cohesion Check Sentence candidates Coherent text Balance and Coverage Coherent text Summary EBM Summarisation Diego Mollá 39/79
51 A Compromise Solution Sentence Extraction Document Sentence candidates This is what most commercial and free summarisers do Cohesion Check Sentence candidates Coherent text Balance and Coverage Coherent text Summary EBM Summarisation Diego Mollá 39/79
52 Contents Evidence Based Medicine What is Evidence Based Medicine? EBM and NLP A Corpus for Summarisation Text Summarisation Sentence Extraction Cohesion Check Balance and Coverage Proposals for Text Summarisation Single-document Summarisation Optimisation and Summarisation EBM Summarisation Diego Mollá 40/79
53 General Approach For each sentence Look for clues to its importance. 2. Compute a score for the sentence based on the clues found. 3. Select all sentences whose scores exceed some threshold. Or select the highest scoring sentences up to a certain total. EBM Summarisation Diego Mollá 41/79
54 The Frequency-keyword Approach 1. Compute the keywords of the document: Ignore the function words by using a stop word list. Sort all remaining words according to frequency or measures such as tf.idf (next slide). Select the top words (say, the top 5%). EBM Summarisation Diego Mollá 42/79
55 The Frequency-keyword Approach 1. Compute the keywords of the document: Ignore the function words by using a stop word list. Sort all remaining words according to frequency or measures such as tf.idf (next slide). Select the top words (say, the top 5%). 2. Score the document sentences according to the presence of keywords: Simple keyword count. Weighted keyword count (keyword weights for each sentence). Looking for keyword clusters in the sentence. EBM Summarisation Diego Mollá 42/79
56 Finding Most Informative Sentences tf.idf to find keywords Term Frequency (tf ): Words that are very frequent in a document are more important. tf (w) = # times wordwis in document Inverse Document Frequency (idf ): Words that appear in many documents are less important. idf (w) = log # documents # documens that contain wordw EBM Summarisation Diego Mollá 43/79
57 The Biased Keyword Approach Title and headings biased Compute a list of keywords on the basis of document structure: select candidates from titles and headings only, or candidates from titles and headings have more importance: e.g. they are counted as being more frequent. EBM Summarisation Diego Mollá 44/79
58 The Biased Keyword Approach Title and headings biased Compute a list of keywords on the basis of document structure: select candidates from titles and headings only, or candidates from titles and headings have more importance: e.g. they are counted as being more frequent. Query biased (customised summaries) Use the user s query to determine the keyword s weights: the user s query determines all the keywords, or the user s query introduces additional keywords or updates the weights of existing keywords. EBM Summarisation Diego Mollá 44/79
59 The Location Method Observation First and last sentence of a paragraph are usually most central to the theme of a text. Increase the score of a sentence according to its position in the paragraph: Beginning of paragraph. End of paragraph. EBM Summarisation Diego Mollá 45/79
60 Cues, Indicator Phrases I Cues Certain words (not necessarily keywords) provide an indication of the importance of the sentence. Use these words to determine the sentence score: bonus words increase the sentence score: greatest, significant stigma words decrease the sentence score: hardly, impossible, now EBM Summarisation Diego Mollá 46/79
61 Cues, Indicator Phrases II Indicator Phrases Indicator phrases are specific phrases or patterns of phrases that can be used to determine the sentence importance: The main aim of the present paper is... The purpose of this article is... In this report, we outline... Our investigation has shown that... EBM Summarisation Diego Mollá 47/79
62 Relational Criteria 1. Build a semantic structure for the document: sentences are vertices inter-sentence links are edges Rhetorical links (ellaboration, sequence, etc) Cooccurrence of keywords Use the link structure to determine the most important sentences Degree of the vertex Eigenvalues (PageRank style) EBM Summarisation Diego Mollá 48/79
63 Contents Evidence Based Medicine What is Evidence Based Medicine? EBM and NLP A Corpus for Summarisation Text Summarisation Sentence Extraction Cohesion Check Balance and Coverage Proposals for Text Summarisation Single-document Summarisation Optimisation and Summarisation EBM Summarisation Diego Mollá 49/79
64 Textual Cohesion Lack of cohesion results in odd extracts. Sentences include references to other sentences: Anaphoric reference: John saw Mary. She was talking over the phone Rhetorical connectives: So, the following example... Lexical or definite reference: I saw a man with a book. The book was... Possible solutions: Aggregation Add preceding sentences until there are no external references. Deletion Remove the difficult sentences. Modification Alter the sentences to eliminate or disguise the problem. EBM Summarisation Diego Mollá 50/79
65 Contents Evidence Based Medicine What is Evidence Based Medicine? EBM and NLP A Corpus for Summarisation Text Summarisation Sentence Extraction Cohesion Check Balance and Coverage Proposals for Text Summarisation Single-document Summarisation Optimisation and Summarisation EBM Summarisation Diego Mollá 51/79
66 Balance and Coverage We need to process the selected sentences in order to produce a real abstract: Delete redundant sentences. Harmonise tense and voice of verbs. Ensure balance and proper coverage. Combination of information extraction and text generation. Need to consider text structure: Each sentence plays a role in the text and in relation with the other sentences. Problem to address: Lack of balance and coverage: Missing important information. Too much emphasis on less important information. EBM Summarisation Diego Mollá 52/79
67 Contents Evidence Based Medicine What is Evidence Based Medicine? EBM and NLP A Corpus for Summarisation Text Summarisation Sentence Extraction Cohesion Check Balance and Coverage Proposals for Text Summarisation Single-document Summarisation Optimisation and Summarisation EBM Summarisation Diego Mollá 53/79
68 Contents Evidence Based Medicine What is Evidence Based Medicine? EBM and NLP A Corpus for Summarisation Text Summarisation Sentence Extraction Cohesion Check Balance and Coverage Proposals for Text Summarisation Single-document Summarisation Optimisation and Summarisation EBM Summarisation Diego Mollá 54/79
69 Single-document Summarisation Input Question. Document Abstract. Output Extractive summary that answers the question. Target summary is the annotated answer justification ( long ). Evaluated using ROUGE-L with Stemming. EBM Summarisation Diego Mollá 55/79
70 General Approach (Sarker et al., CBMS 2012) In a Nutshell 1. Gather statistics from the best 3-sentence extracts. Exhaustive search to find these best extracts. 2. Build three classifiers, one per sentence in the final extract. Classifier 1 based on statistics from best 1st sentence. Classifier 2 based on statistics from best 2nd sentence. Classifier 3 based on statistics from best 3rd sentence. EBM Summarisation Diego Mollá 56/79
71 The Statistics Gathered 1. Source sentence position. 2. Sentence length. 3. Sentence similarity. 4. Sentence type. EBM Summarisation Diego Mollá 57/79
72 1. Source Sentence Position Compute relative positions (0... 1). Create normalised frequency histograms f 1, f 2,..., f 10. Score every relative position in bin i with its bin frequency: S pos (i) = f bin(i). EBM Summarisation Diego Mollá 58/79
73 2. Sentence Length Reward larger sentences and penalise shorter sentences: Normalised sentence length l s : sentence length S len (i) = l s l avg l d l avg : average sentence length in the corpus l d : document length EBM Summarisation Diego Mollá 59/79
74 3. Sentence Similarity Sentence Similarity Lowercase, stem, remove stop words. Build vector of tf.idf with remaining words and UMLS semantic types. CosSim(X, Y ) = X.Y X Y Maximal Marginal Relevance (Carbonell & Goldstein, 1998) Reward sentences similar to the query and penalise those similar to other summary sentences. MMR = λ(cossim(s i, Q)) (1 λ)max Sj ɛs(cossim(s i, S j )) EBM Summarisation Diego Mollá 60/79
75 4. PIBOSO (Kim et al. 2011) I 1. Classify all sentences into PIBOSO types (a variant of PICO). 2. Generate normalised frequency histograms of resulting PIBOSO types. EBM Summarisation Diego Mollá 61/79
76 4. PIBOSO (Kim et al. 2011) II Position independent S PIPS (i) = P best P all Position dependent S PDPS (i) = Ppos P best P best : proportion of this PIBOSO type among all best summary sentences. P all : proportion of this PIBOSO type among all sentences. P pos: proportion of this PIBOSO type among all best summary sentences at this position. EBM Summarisation Diego Mollá 62/79
77 Classification Edmunsonian Formula S Si = αs rposi + βs leni + γs PIPSi +δs PDPSi + ɛs MMRi MMR is replaced with cosine similarity for first sentence. In case of ties, the sentence with greatest length is chosen. Parameters are fine-tuned through exhaustive search (grid search) using training set. α = 1.0, β = 0.8, γ = 0.1, δ = 0.8, ɛ = 0.1, λ = 0.1. EBM Summarisation Diego Mollá 63/79
78 Percentile-based Evaluation (Ceylan et al. 2010) I We compare against all possible 3-sentence extracts in the test set. 1. Bin all possible three-sentence combinations of each abstract. 1,000 bins. 2. Normalise the resulting histograms. 3. Combine all histograms. convolution. 4. The result approximates the probability density distribution of all three-sentence summaries in all abstracts. EBM Summarisation Diego Mollá 64/79
79 Percentile-based Evaluation (Ceylan et al. 2010) II EBM Summarisation Diego Mollá 65/79
80 Systems L3 Last three sentences. O3 Last three PIBOSO outcome sentences. R Random. O All outcome sentences. PI Sentence position independent. PD Sentence position dependent (our proposal). EBM Summarisation Diego Mollá 66/79
81 Results System F-Score 95% CI Percentile (%) L O R O PI PD EBM Summarisation Diego Mollá 67/79
82 Towards Multi-document Summarisation Evidence suggests that a two-step process is promising (Sarker et al., ALTA 2012). 1. Single document summarisation. 2. Multi-document summarisation from the single-document summaries. Traditional clustering techniques seem to produce good clustering of references (Shash & Molla, unpublished). We are still looking at means to obtain the answer parts. Topics as cluster centroids. Overlap with the question. EBM Summarisation Diego Mollá 68/79
83 Contents Evidence Based Medicine What is Evidence Based Medicine? EBM and NLP A Corpus for Summarisation Text Summarisation Sentence Extraction Cohesion Check Balance and Coverage Proposals for Text Summarisation Single-document Summarisation Optimisation and Summarisation EBM Summarisation Diego Mollá 69/79
84 Optimisation and NLP Many NLP tasks are based on optimisation Text classification: minimise the classification error. Part of speech tagging: Find the optimal sequence of labels. Parsing: Find the most likely parse. Machine translation: Dual optimisation. The target sentence must keep the most meaning. The target sentence must try to follow the language model of the target language. EBM Summarisation Diego Mollá 70/79
85 Optimisation in Machine Learning Many ML tasks are about optimising parameters arg min θ J(h θ (X ), Y ) J Cost (error) function. θ Machine Learning parametres. h θ (X ) Hypothesis function. X Inputs. Y Observed results. EBM Summarisation Diego Mollá 71/79
86 Optimisation in Classification Tasks Hypothesis function in Logistic Regression Sigmoid (logistic) function h θ (X ) = e i θ i x i = e θt X Cost function in Logistic Regression Cross entropy [ m ] J(h θ (X ), Y ) = 1 m i=1 y (i) log(h θ (x (i) ) + (1 y (i) ) log(1 h θ (x (i) )) EBM Summarisation Diego Mollá 72/79
87 Query Based Summarisation I A summary sentence S i must maximise its similarity with the question arg max S CosSim(S i, Q) i A summary sentence S i must minimise its similarity with other summary sentences arg min S CosSim(S i, S j ) i,j EBM Summarisation Diego Mollá 73/79
88 Query Based Summarisation II Maximal Marginal Relevance (Carbonell & Goldstein, 1998) Greedy approach Each iteration, select the sentence S i with highest MMR score. MMR = λ(cossim(s i, Q)) (1 λ) max Sj ɛs(cossim(s i, S j )) EBM Summarisation Diego Mollá 74/79
89 Single-document Summarisation Information contents in summary sentences must be maximal The sum of all weighted concepts in a summary must be maximal. arg max w(c) S c C S This is the knapsack problem (NP-hard). Readability in summary sentences must be maximal We ve seen some aspects of readability above now we need to express them as a problem of optimisation. EBM Summarisation Diego Mollá 75/79
90 Query-based Multi-document Abstractive Summarisation The summary must fit the question topics best 1. Fitting to cluster centroids. 2. Topic modelling (LDA) variants. The summary must be most readable Find the word extracts that are most likely produced by language models: 1. Word sequences (e.g. 2-grams). 2. Best likely parse (e.g. the k-minimum spanning tree in a graph). EBM Summarisation Diego Mollá 76/79
91 Latent Dirichlet Allocation N D Π d Z dn α w dn θ k β K Π d : topic probability distribution for document d. Z dn : actual topic selected for word n in document d. θ k : word probability distribution for topic k. α, β: hyperparameters. EBM Summarisation Diego Mollá 77/79
92 Possible LDA Variant for Query-focused Summarisation q r π z θ q w θ d questions words documents doc t EBM Summarisation Diego Mollá 78/79
93 Summary Evidence Based Medicine (EBM) is an important problem that medical doctors face. EBM can benefit from Natural Language Processing (NLP) in general, and text summarisation in particular. Text summarisation, like many NLP tasks, relies on optimisation. We need expertise on optimisation techniques! EBM Summarisation Diego Mollá 79/79
94 Summary Evidence Based Medicine (EBM) is an important problem that medical doctors face. EBM can benefit from Natural Language Processing (NLP) in general, and text summarisation in particular. Text summarisation, like many NLP tasks, relies on optimisation. We need expertise on optimisation techniques! Questions? Further information about our research: EBM Summarisation Diego Mollá 79/79
Automated Summarisation for Evidence Based Medicine
Automated Summarisation for Evidence Based Medicine Diego Mollá Centre for Language Technology, Macquarie University HAIL, 22 March 2012 Contents Evidence Based Medicine Our Corpus for Summarisation Structure
Lab 12: Structured Prediction
December 4, 2014 Lecture plan structured perceptron application: confused messages application: dependency parsing structured SVM Class review: from modelization to classification What does learning mean?
Part A. P (w 1 )P (w 2 w 1 )P (w 3 w 1 w 2 ) P (w M w 1 w 2 w M 1 ) P (w 1 )P (w 2 w 1 )P (w 3 w 2 ) P (w M w M 1 )
Part A 1. A Markov chain is a discrete-time stochastic process, defined by a set of states, a set of transition probabilities (between states), and a set of initial state probabilities; the process proceeds
Sparse vectors recap. ANLP Lecture 22 Lexical Semantics with Dense Vectors. Before density, another approach to normalisation.
ANLP Lecture 22 Lexical Semantics with Dense Vectors Henry S. Thompson Based on slides by Jurafsky & Martin, some via Dorota Glowacka 5 November 2018 Previous lectures: Sparse vectors recap How to represent
ANLP Lecture 22 Lexical Semantics with Dense Vectors
ANLP Lecture 22 Lexical Semantics with Dense Vectors Henry S. Thompson Based on slides by Jurafsky & Martin, some via Dorota Glowacka 5 November 2018 Henry S. Thompson ANLP Lecture 22 5 November 2018 Previous
Part I: Web Structure Mining Chapter 1: Information Retrieval and Web Search
Part I: Web Structure Mining Chapter : Information Retrieval an Web Search The Web Challenges Crawling the Web Inexing an Keywor Search Evaluating Search Quality Similarity Search The Web Challenges Tim
Generic Text Summarization
June 27, 2012 Outline Introduction 1 Introduction Notation and Terminology 2 3 4 5 6 Text Summarization Introduction Notation and Terminology Two Types of Text Summarization Query-Relevant Summarization:
Click Prediction and Preference Ranking of RSS Feeds
Click Prediction and Preference Ranking of RSS Feeds 1 Introduction December 11, 2009 Steven Wu RSS (Really Simple Syndication) is a family of data formats used to publish frequently updated works. RSS
Boolean and Vector Space Retrieval Models
Boolean and Vector Space Retrieval Models Many slides in this section are adapted from Prof. Joydeep Ghosh (UT ECE) who in turn adapted them from Prof. Dik Lee (Univ. of Science and Tech, Hong Kong) 1
Text Mining. Dr. Yanjun Li. Associate Professor. Department of Computer and Information Sciences Fordham University
Text Mining Dr. Yanjun Li Associate Professor Department of Computer and Information Sciences Fordham University Outline Introduction: Data Mining Part One: Text Mining Part Two: Preprocessing Text Data
Generative Clustering, Topic Modeling, & Bayesian Inference
Generative Clustering, Topic Modeling, & Bayesian Inference INFO-4604, Applied Machine Learning University of Colorado Boulder December 12-14, 2017 Prof. Michael Paul Unsupervised Naïve Bayes Last week
Modern Information Retrieval
Modern Information Retrieval Chapter 8 Text Classification Introduction A Characterization of Text Classification Unsupervised Algorithms Supervised Algorithms Feature Selection or Dimensionality Reduction
Topic Models and Applications to Short Documents
Topic Models and Applications to Short Documents Dieu-Thu Le Email: dieuthu.le@unitn.it Trento University April 6, 2011 1 / 43 Outline Introduction Latent Dirichlet Allocation Gibbs Sampling Short Text
Language Models. CS6200: Information Retrieval. Slides by: Jesse Anderton
Language Models CS6200: Information Retrieval Slides by: Jesse Anderton What s wrong with VSMs? Vector Space Models work reasonably well, but have a few problems: They are based on bag-of-words, so they
Improving Diversity in Ranking using Absorbing Random Walks
Improving Diversity in Ranking using Absorbing Random Walks Andrew B. Goldberg with Xiaojin Zhu, Jurgen Van Gael, and David Andrzejewski Department of Computer Sciences, University of Wisconsin, Madison
1 Information retrieval fundamentals
CS 630 Lecture 1: 01/26/2006 Lecturer: Lillian Lee Scribes: Asif-ul Haque, Benyah Shaparenko This lecture focuses on the following topics Information retrieval fundamentals Vector Space Model (VSM) Deriving
Natural Language Processing. Topics in Information Retrieval. Updated 5/10
Natural Language Processing Topics in Information Retrieval Updated 5/10 Outline Introduction to IR Design features of IR systems Evaluation measures The vector space model Latent semantic indexing Background
Text mining and natural language analysis. Jefrey Lijffijt
Text mining and natural language analysis Jefrey Lijffijt PART I: Introduction to Text Mining Why text mining The amount of text published on paper, on the web, and even within companies is inconceivably
Chap 2: Classical models for information retrieval
Chap 2: Classical models for information retrieval Jean-Pierre Chevallet & Philippe Mulhem LIG-MRIM Sept 2016 Jean-Pierre Chevallet & Philippe Mulhem Models of IR 1 / 81 Outline Basic IR Models 1 Basic
Language as a Stochastic Process
CS769 Spring 2010 Advanced Natural Language Processing Language as a Stochastic Process Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu 1 Basic Statistics for NLP Pick an arbitrary letter x at random from any
DISTRIBUTIONAL SEMANTICS
COMP90042 LECTURE 4 DISTRIBUTIONAL SEMANTICS LEXICAL DATABASES - PROBLEMS Manually constructed Expensive Human annotation can be biased and noisy Language is dynamic New words: slangs, terminology, etc.
Machine Learning (CS 567) Lecture 2
Machine Learning (CS 567) Lecture 2 Time: T-Th 5:00pm - 6:20pm Location: GFS118 Instructor: Sofus A. Macskassy (macskass@usc.edu) Office: SAL 216 Office hours: by appointment Teaching assistant: Cheol
Prediction of Citations for Academic Papers
000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Medical Question Answering for Clinical Decision Support
Medical Question Answering for Clinical Decision Support Travis R. Goodwin and Sanda M. Harabagiu The University of Texas at Dallas Human Language Technology Research Institute http://www.hlt.utdallas.edu
NEAL: A Neurally Enhanced Approach to Linking Citation and Reference
NEAL: A Neurally Enhanced Approach to Linking Citation and Reference Tadashi Nomoto 1 National Institute of Japanese Literature 2 The Graduate University of Advanced Studies (SOKENDAI) nomoto@acm.org Abstract.
Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen. Language Models. Tobias Scheffer
Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Language Models Tobias Scheffer Stochastic Language Models A stochastic language model is a probability distribution over words.
INFO 4300 / CS4300 Information Retrieval. slides adapted from Hinrich Schütze s, linked from
INFO 4300 / CS4300 Information Retrieval slides adapted from Hinrich Schütze s, linked from http://informationretrieval.org/ IR 26/26: Feature Selection and Exam Overview Paul Ginsparg Cornell University,
Generative Models for Sentences
Generative Models for Sentences Amjad Almahairi PhD student August 16 th 2014 Outline 1. Motivation Language modelling Full Sentence Embeddings 2. Approach Bayesian Networks Variational Autoencoders (VAE)
Part of Speech Tagging: Viterbi, Forward, Backward, Forward- Backward, Baum-Welch. COMP-599 Oct 1, 2015
Part of Speech Tagging: Viterbi, Forward, Backward, Forward- Backward, Baum-Welch COMP-599 Oct 1, 2015 Announcements Research skills workshop today 3pm-4:30pm Schulich Library room 313 Start thinking about
Statistical NLP for the Web Log Linear Models, MEMM, Conditional Random Fields
Statistical NLP for the Web Log Linear Models, MEMM, Conditional Random Fields Sameer Maskey Week 13, Nov 28, 2012 1 Announcements Next lecture is the last lecture Wrap up of the semester 2 Final Project
Formal Ontology Construction from Texts
Formal Ontology Construction from Texts Yue MA Theoretical Computer Science Faculty Informatics TU Dresden June 16, 2014 Outline 1 Introduction Ontology: definition and examples 2 Ontology Construction
Multi-theme Sentiment Analysis using Quantified Contextual
Multi-theme Sentiment Analysis using Quantified Contextual Valence Shifters Hongkun Yu, Jingbo Shang, MeichunHsu, Malú Castellanos, Jiawei Han Presented by Jingbo Shang University of Illinois at Urbana-Champaign
Linear Classifiers IV
Universität Potsdam Institut für Informatik Lehrstuhl Linear Classifiers IV Blaine Nelson, Tobias Scheffer Contents Classification Problem Bayesian Classifier Decision Linear Classifiers, MAP Models Logistic
Natural Language Processing Prof. Pawan Goyal Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur
Natural Language Processing Prof. Pawan Goyal Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Lecture - 18 Maximum Entropy Models I Welcome back for the 3rd module
Math for Machine Learning Open Doors to Data Science and Artificial Intelligence. Richard Han
Math for Machine Learning Open Doors to Data Science and Artificial Intelligence Richard Han Copyright 05 Richard Han All rights reserved. CONTENTS PREFACE... - INTRODUCTION... LINEAR REGRESSION... 4 LINEAR
Algorithms Exam TIN093 /DIT602
Algorithms Exam TIN093 /DIT602 Course: Algorithms Course code: TIN 093, TIN 092 (CTH), DIT 602 (GU) Date, time: 21st October 2017, 14:00 18:00 Building: SBM Responsible teacher: Peter Damaschke, Tel. 5405
CS145: INTRODUCTION TO DATA MINING
CS145: INTRODUCTION TO DATA MINING Text Data: Topic Model Instructor: Yizhou Sun yzsun@cs.ucla.edu December 4, 2017 Methods to be Learnt Vector Data Set Data Sequence Data Text Data Classification Clustering
Boolean and Vector Space Retrieval Models CS 290N Some of slides from R. Mooney (UTexas), J. Ghosh (UT ECE), D. Lee (USTHK).
Boolean and Vector Space Retrieval Models 2013 CS 290N Some of slides from R. Mooney (UTexas), J. Ghosh (UT ECE), D. Lee (USTHK). 1 Table of Content Boolean model Statistical vector space model Retrieval
From perceptrons to word embeddings. Simon Šuster University of Groningen
From perceptrons to word embeddings Simon Šuster University of Groningen Outline A basic computational unit Weighting some input to produce an output: classification Perceptron Classify tweets Written
Statistical Ranking Problem
Statistical Ranking Problem Tong Zhang Statistics Department, Rutgers University Ranking Problems Rank a set of items and display to users in corresponding order. Two issues: performance on top and dealing
RETRIEVAL MODELS. Dr. Gjergji Kasneci Introduction to Information Retrieval WS
RETRIEVAL MODELS Dr. Gjergji Kasneci Introduction to Information Retrieval WS 2012-13 1 Outline Intro Basics of probability and information theory Retrieval models Boolean model Vector space model Probabilistic
SocViz: Visualization of Facebook Data
SocViz: Visualization of Facebook Data Abhinav S Bhatele Department of Computer Science University of Illinois at Urbana Champaign Urbana, IL 61801 USA bhatele2@uiuc.edu Kyratso Karahalios Department of
N-grams. Motivation. Simple n-grams. Smoothing. Backoff. N-grams L545. Dept. of Linguistics, Indiana University Spring / 24
L545 Dept. of Linguistics, Indiana University Spring 2013 1 / 24 Morphosyntax We just finished talking about morphology (cf. words) And pretty soon we re going to discuss syntax (cf. sentences) In between,
CSE 494/598 Lecture-4: Correlation Analysis. **Content adapted from last year s slides
CSE 494/598 Lecture-4: Correlation Analysis LYDIA MANIKONDA HT TP://WWW.PUBLIC.ASU.EDU/~LMANIKON / **Content adapted from last year s slides Announcements Project-1 Due: February 12 th 2016 Analysis report:
Natural Language Processing
Natural Language Processing Word vectors Many slides borrowed from Richard Socher and Chris Manning Lecture plan Word representations Word vectors (embeddings) skip-gram algorithm Relation to matrix factorization
Statistical Machine Learning Theory. From Multi-class Classification to Structured Output Prediction. Hisashi Kashima.
http://goo.gl/jv7vj9 Course website KYOTO UNIVERSITY Statistical Machine Learning Theory From Multi-class Classification to Structured Output Prediction Hisashi Kashima kashima@i.kyoto-u.ac.jp DEPARTMENT
Information Retrieval. Lecture 6
Information Retrieval Lecture 6 Recap of the last lecture Parametric and field searches Zones in documents Scoring documents: zone weighting Index support for scoring tf idf and vector spaces This lecture
Ranked Retrieval (2)
Text Technologies for Data Science INFR11145 Ranked Retrieval (2) Instructor: Walid Magdy 31-Oct-2017 Lecture Objectives Learn about Probabilistic models BM25 Learn about LM for IR 2 1 Recall: VSM & TFIDF
Tuning as Linear Regression
Tuning as Linear Regression Marzieh Bazrafshan, Tagyoung Chung and Daniel Gildea Department of Computer Science University of Rochester Rochester, NY 14627 Abstract We propose a tuning method for statistical
Comparative Document Analysis for Large Text Corpora
Comparative Document Analysis for Large Text Corpora Xiang Ren Yuanhua Lv Kuansan Wang Jiawei Han University of Illinois at Urbana-Champaign, Urbana, IL, USA Microsoft Research, Redmond, WA, USA {xren7,
Long-Short Term Memory and Other Gated RNNs
Long-Short Term Memory and Other Gated RNNs Sargur Srihari srihari@buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Topics in Sequence Modeling
Statistical Machine Learning Theory. From Multi-class Classification to Structured Output Prediction. Hisashi Kashima.
http://goo.gl/xilnmn Course website KYOTO UNIVERSITY Statistical Machine Learning Theory From Multi-class Classification to Structured Output Prediction Hisashi Kashima kashima@i.kyoto-u.ac.jp DEPARTMENT
Formal Modeling in Cognitive Science
Formal Modeling in Cognitive Science Lecture 9: Application of Bayes Theorem; Discrete Random Variables; Steve Renals (notes by Frank Keller) School of Informatics University of Edinburgh s.renals@ed.ac.uk
Formal Modeling in Cognitive Science Lecture 19: Application of Bayes Theorem; Discrete Random Variables; Distributions. Background.
Formal Modeling in Cognitive Science Lecture 9: ; Discrete Random Variables; Steve Renals (notes by Frank Keller) School of Informatics University of Edinburgh s.renals@ed.ac.uk February 7 Probability
Gaussian Models
Gaussian Models ddebarr@uw.edu 2016-04-28 Agenda Introduction Gaussian Discriminant Analysis Inference Linear Gaussian Systems The Wishart Distribution Inferring Parameters Introduction Gaussian Density
More Smoothing, Tuning, and Evaluation
More Smoothing, Tuning, and Evaluation Nathan Schneider (slides adapted from Henry Thompson, Alex Lascarides, Chris Dyer, Noah Smith, et al.) ENLP 21 September 2016 1 Review: 2 Naïve Bayes Classifier w
Information Retrieval and Topic Models. Mausam (Based on slides of W. Arms, Dan Jurafsky, Thomas Hofmann, Ata Kaban, Chris Manning, Melanie Martin)
Information Retrieval and Topic Models Mausam (Based on slides of W. Arms, Dan Jurafsky, Thomas Hofmann, Ata Kaban, Chris Manning, Melanie Martin) Sec. 1.1 Unstructured data in 1620 Which plays of Shakespeare
Lecture 5 Neural models for NLP
CS546: Machine Learning in NLP (Spring 2018) http://courses.engr.illinois.edu/cs546/ Lecture 5 Neural models for NLP Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center Office hours: Tue/Thu 2pm-3pm
9/12/17. Types of learning. Modeling data. Supervised learning: Classification. Supervised learning: Regression. Unsupervised learning: Clustering
Types of learning Modeling data Supervised: we know input and targets Goal is to learn a model that, given input data, accurately predicts target data Unsupervised: we know the input only and want to make
Vector-based Models of Semantic Composition. Jeff Mitchell and Mirella Lapata, 2008
Vector-based Models of Semantic Composition Jeff Mitchell and Mirella Lapata, 2008 Composition in Distributional Models of Semantics Jeff Mitchell and Mirella Lapata, 2010 Distributional Hypothesis Words
Learning Features from Co-occurrences: A Theoretical Analysis
Learning Features from Co-occurrences: A Theoretical Analysis Yanpeng Li IBM T. J. Watson Research Center Yorktown Heights, New York 10598 liyanpeng.lyp@gmail.com Abstract Representing a word by its co-occurrences
CSE 546 Final Exam, Autumn 2013
CSE 546 Final Exam, Autumn 0. Personal info: Name: Student ID: E-mail address:. There should be 5 numbered pages in this exam (including this cover sheet).. You can use any material you brought: any book,
Retrieval by Content. Part 2: Text Retrieval Term Frequency and Inverse Document Frequency. Srihari: CSE 626 1
Retrieval by Content Part 2: Text Retrieval Term Frequency and Inverse Document Frequency Srihari: CSE 626 1 Text Retrieval Retrieval of text-based information is referred to as Information Retrieval (IR)
The Noisy Channel Model and Markov Models
1/24 The Noisy Channel Model and Markov Models Mark Johnson September 3, 2014 2/24 The big ideas The story so far: machine learning classifiers learn a function that maps a data item X to a label Y handle
On the Foundations of Diverse Information Retrieval. Scott Sanner, Kar Wai Lim, Shengbo Guo, Thore Graepel, Sarvnaz Karimi, Sadegh Kharazmi
On the Foundations of Diverse Information Retrieval Scott Sanner, Kar Wai Lim, Shengbo Guo, Thore Graepel, Sarvnaz Karimi, Sadegh Kharazmi 1 Outline Need for diversity The answer: MMR But what was the
Collaborative Topic Modeling for Recommending Scientific Articles
Collaborative Topic Modeling for Recommending Scientific Articles Chong Wang and David M. Blei Best student paper award at KDD 2011 Computer Science Department, Princeton University Presented by Tian Cao
Low-Dimensional Discriminative Reranking. Jagadeesh Jagarlamudi and Hal Daume III University of Maryland, College Park
Low-Dimensional Discriminative Reranking Jagadeesh Jagarlamudi and Hal Daume III University of Maryland, College Park Discriminative Reranking Useful for many NLP tasks Enables us to use arbitrary features
MSA220/MVE440 Statistical Learning for Big Data
MSA220/MVE440 Statistical Learning for Big Data Lecture 7/8 - High-dimensional modeling part 1 Rebecka Jörnsten Mathematical Sciences University of Gothenburg and Chalmers University of Technology Classification
Machine Learning for NLP
Machine Learning for NLP Uppsala University Department of Linguistics and Philology Slides borrowed from Ryan McDonald, Google Research Machine Learning for NLP 1(50) Introduction Linear Classifiers Classifiers
Information Retrieval
Introduction to Information CS276: Information and Web Search Christopher Manning and Pandu Nayak Lecture 13: Latent Semantic Indexing Ch. 18 Today s topic Latent Semantic Indexing Term-document matrices
Applied Natural Language Processing
Applied Natural Language Processing Info 256 Lecture 7: Testing (Feb 12, 2019) David Bamman, UC Berkeley Significance in NLP You develop a new method for text classification; is it better than what comes
CS276A Text Information Retrieval, Mining, and Exploitation. Lecture 4 15 Oct 2002
CS276A Text Information Retrieval, Mining, and Exploitation Lecture 4 15 Oct 2002 Recap of last time Index size Index construction techniques Dynamic indices Real world considerations 2 Back of the envelope
Applications of Submodular Functions in Speech and NLP
Applications of Submodular Functions in Speech and NLP Jeff Bilmes Department of Electrical Engineering University of Washington, Seattle http://ssli.ee.washington.edu/~bilmes June 27, 2011 J. Bilmes Applications
Log-Linear Models, MEMMs, and CRFs
Log-Linear Models, MEMMs, and CRFs Michael Collins 1 Notation Throughout this note I ll use underline to denote vectors. For example, w R d will be a vector with components w 1, w 2,... w d. We use expx
Midterm sample questions
Midterm sample questions CS 585, Brendan O Connor and David Belanger October 12, 2014 1 Topics on the midterm Language concepts Translation issues: word order, multiword translations Human evaluation Parts
Lecture 13: Structured Prediction
Lecture 13: Structured Prediction Kai-Wei Chang CS @ University of Virginia kw@kwchang.net Couse webpage: http://kwchang.net/teaching/nlp16 CS6501: NLP 1 Quiz 2 v Lectures 9-13 v Lecture 12: before page
Data Mining Techniques
Data Mining Techniques CS 622 - Section 2 - Spring 27 Pre-final Review Jan-Willem van de Meent Feedback Feedback https://goo.gl/er7eo8 (also posted on Piazza) Also, please fill out your TRACE evaluations!
CPSC 340: Machine Learning and Data Mining
CPSC 340: Machine Learning and Data Mining MLE and MAP Original version of these slides by Mark Schmidt, with modifications by Mike Gelbart. 1 Admin Assignment 4: Due tonight. Assignment 5: Will be released
Comparative Summarization via Latent Dirichlet Allocation
Comparative Summarization via Latent Dirichlet Allocation Michal Campr and Karel Jezek Department of Computer Science and Engineering, FAV, University of West Bohemia, 11 February 2013, 301 00, Plzen,
Latent Dirichlet Allocation Introduction/Overview
Latent Dirichlet Allocation Introduction/Overview David Meyer 03.10.2016 David Meyer http://www.1-4-5.net/~dmm/ml/lda_intro.pdf 03.10.2016 Agenda What is Topic Modeling? Parametric vs. Non-Parametric Models
NATURAL LANGUAGE PROCESSING. Dr. G. Bharadwaja Kumar
NATURAL LANGUAGE PROCESSING Dr. G. Bharadwaja Kumar Sentence Boundary Markers Many natural language processing (NLP) systems generally take a sentence as an input unit part of speech (POS) tagging, chunking,
Behavioral Data Mining. Lecture 2
Behavioral Data Mining Lecture 2 Autonomy Corp Bayes Theorem Bayes Theorem P(A B) = probability of A given that B is true. P(A B) = P(B A)P(A) P(B) In practice we are most interested in dealing with events
Sampling for Frequent Itemset Mining The Power of PAC learning
Sampling for Frequent Itemset Mining The Power of PAC learning prof. dr Arno Siebes Algorithmic Data Analysis Group Department of Information and Computing Sciences Universiteit Utrecht The Papers Today
Decision Trees. Nicholas Ruozzi University of Texas at Dallas. Based on the slides of Vibhav Gogate and David Sontag
Decision Trees Nicholas Ruozzi University of Texas at Dallas Based on the slides of Vibhav Gogate and David Sontag Supervised Learning Input: labelled training data i.e., data plus desired output Assumption:
CS 188: Artificial Intelligence Spring Today
CS 188: Artificial Intelligence Spring 2006 Lecture 9: Naïve Bayes 2/14/2006 Dan Klein UC Berkeley Many slides from either Stuart Russell or Andrew Moore Bayes rule Today Expectations and utilities Naïve
Penn Treebank Parsing. Advanced Topics in Language Processing Stephen Clark
Penn Treebank Parsing Advanced Topics in Language Processing Stephen Clark 1 The Penn Treebank 40,000 sentences of WSJ newspaper text annotated with phrasestructure trees The trees contain some predicate-argument
A brief introduction to Conditional Random Fields
A brief introduction to Conditional Random Fields Mark Johnson Macquarie University April, 2005, updated October 2010 1 Talk outline Graphical models Maximum likelihood and maximum conditional likelihood
Predicting New Search-Query Cluster Volume
Predicting New Search-Query Cluster Volume Jacob Sisk, Cory Barr December 14, 2007 1 Problem Statement Search engines allow people to find information important to them, and search engine companies derive
Lecture 6: Neural Networks for Representing Word Meaning
Lecture 6: Neural Networks for Representing Word Meaning Mirella Lapata School of Informatics University of Edinburgh mlap@inf.ed.ac.uk February 7, 2017 1 / 28 Logistic Regression Input is a feature vector,
Introduction to AI Learning Bayesian networks. Vibhav Gogate
Introduction to AI Learning Bayesian networks Vibhav Gogate Inductive Learning in a nutshell Given: Data Examples of a function (X, F(X)) Predict function F(X) for new examples X Discrete F(X): Classification
Natural Language Processing. Classification. Features. Some Definitions. Classification. Feature Vectors. Classification I. Dan Klein UC Berkeley
Natural Language Processing Classification Classification I Dan Klein UC Berkeley Classification Automatically make a decision about inputs Example: document category Example: image of digit digit Example:
Probabilistic Graphical Models: MRFs and CRFs. CSE628: Natural Language Processing Guest Lecturer: Veselin Stoyanov
Probabilistic Graphical Models: MRFs and CRFs CSE628: Natural Language Processing Guest Lecturer: Veselin Stoyanov Why PGMs? PGMs can model joint probabilities of many events. many techniques commonly
Special Topics in Computer Science
Special Topics in Computer Science NLP in a Nutshell CS492B Spring Semester 2009 Speaker : Hee Jin Lee p Professor : Jong C. Park Computer Science Department Korea Advanced Institute of Science and Technology
CS460/626 : Natural Language
CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 23, 24 Parsing Algorithms; Parsing in case of Ambiguity; Probabilistic Parsing) Pushpak Bhattacharyya CSE Dept., IIT Bombay 8 th,
Lecture 12: Algorithms for HMMs
Lecture 12: Algorithms for HMMs Nathan Schneider (some slides from Sharon Goldwater; thanks to Jonathan May for bug fixes) ENLP 26 February 2018 Recap: tagging POS tagging is a sequence labelling task.
Outline: Ensemble Learning. Ensemble Learning. The Wisdom of Crowds. The Wisdom of Crowds - Really? Crowd wiser than any individual
Outline: Ensemble Learning We will describe and investigate algorithms to Ensemble Learning Lecture 10, DD2431 Machine Learning A. Maki, J. Sullivan October 2014 train weak classifiers/regressors and how
Spatial Role Labeling CS365 Course Project
Spatial Role Labeling CS365 Course Project Amit Kumar, akkumar@iitk.ac.in Chandra Sekhar, gchandra@iitk.ac.in Supervisor : Dr.Amitabha Mukerjee ABSTRACT In natural language processing one of the important
CSE 417T: Introduction to Machine Learning. Final Review. Henry Chai 12/4/18
CSE 417T: Introduction to Machine Learning Final Review Henry Chai 12/4/18 Overfitting Overfitting is fitting the training data more than is warranted Fitting noise rather than signal 2 Estimating! "#$
Lecture 7: DecisionTrees
Lecture 7: DecisionTrees What are decision trees? Brief interlude on information theory Decision tree construction Overfitting avoidance Regression trees COMP-652, Lecture 7 - September 28, 2009 1 Recall:
COMS F18 Homework 3 (due October 29, 2018)
COMS 477-2 F8 Homework 3 (due October 29, 208) Instructions Submit your write-up on Gradescope as a neatly typeset (not scanned nor handwritten) PDF document by :59 PM of the due date. On Gradescope, be