Exploring Content Models for Multi-Document Summarization
|
|
|
- Quentin Hodge
- 5 years ago
- Views:
Transcription
1 Exploring Content Models for Multi-Document Summarization Aria Haghighi UC Berkeley Lucy Vanderwende MSR Redmond
2 Summarization
3 Summarization D
4 Summarization D S
5 Sentence Extraction D S
6 Sentence Extraction D S
7 Summarization
8 Summarization Representation P C ( ) Sotomayor: 0.16 court: 0.13 supreme: 0.13 nominee:
9 Summarization Representation Extraction P C ( ) Sotomayor: 0.16 court: 0.13 supreme: 0.13 nominee:
10 SumBasic: Representation [Nenkova & Vanderwende] 2006
11 SumBasic: Representation Simple Unigram MLE [Nenkova & Vanderwende] 2006
12 SumBasic: Representation Simple Unigram MLE P C (w) = ˆP D (w) Sotomayor: 0.15 Washington: 0.13 supreme: 0.12 Obama: [Nenkova & Vanderwende] 2006
13 SumBasic: Extraction Score(S) = 1 S w S P C(w)
14 SumBasic: Extraction Sentence Score Score(S) = 1 S w S P C(w)
15 SumBasic: Extraction Sentence Score Score(S) = 1 S w S P C(w) Obama announced the nomination of Sonia Sotomayor
16 SumBasic: Extraction Sentence Score Score(S) = 1 S w S P C(w) Obama announced the nomination of Sonia Sotomayor
17 SumBasic: Extraction Sentence Score Score(S) = 1 S w S P C(w) Score: Obama announced the nomination of Sonia Sotomayor
18 SumBasic: Extraction Score(S) = 1 S w S P C(w) S = {}
19 SumBasic: Extraction Score(S) = 1 S w S P C(w) S = {} while words(s) <L:
20 SumBasic: Extraction Score(S) = 1 S w S P C(w) S = {} while words(s) <L: S = max S/ S Score(S)
21 SumBasic: Extraction Score(S) = 1 S w S P C(w) S = {} while words(s) <L: S = max S/ S Score(S) S = S S
22 SumBasic: Extraction Score(S) = 1 S w S P C(w) S = {} while words(s) <L: S = max S/ S Score(S) S = S S P C (w) =P C (w) 2, for w S
23 Experimental Results
24 Experimental Results DUC 2006
25 Experimental Results DUC document sets, 25 docs each
26 Experimental Results DUC document sets, 25 docs each Max 250 tokens
27 Experimental Results DUC document sets, 25 docs each Max 250 tokens ROUGE-2
28 Experimental Results DUC document sets, 25 docs each Max 250 tokens ROUGE-2 Recall over bigrams w/o stop words against human summaries
29 Experimental Results DUC document sets, 25 docs each Max 250 tokens ROUGE-2 Recall over bigrams w/o stop words against human summaries Bad at summary quality, decent at content
30 SumBasic: Performance SumBasic
31 SumBasic: Extraction Issues
32 SumBasic: Extraction Issues What are we optimizing?
33 SumBasic: Extraction Issues What are we optimizing? Without word length, when to stop?
34 SumBasic: Extraction Issues What are we optimizing? Without word length, when to stop? Not Recall Oriented
35 SumBasic: Extraction Issues What are we optimizing? Without word length, when to stop? Not Recall Oriented No direct penalty for missing freq. words
36 KLSum: Extraction P C ( ) Sotomayor: 0.15 Washington: 0.13 supreme: 0.12
37 KLSum: Extraction P C ( ) Sotomayor: 0.15 Washington: 0.13 supreme: 0.12
38 KLSum: Extraction P C ( ) Sotomayor: 0.15 Washington: 0.13 supreme: 0.12 P S ( ) Sotomayor: 0.20 Obama: 0.14 Washington: 0.11
39 KLSum: Extraction P C ( ) Sotomayor: 0.15 Washington: 0.13 supreme: 0.12 P S ( ) Sotomayor: 0.20 Obama: 0.14 Washington: 0.11
40 KLSum: Extraction P C ( ) Sotomayor: 0.15 Washington: 0.13 supreme: 0.12 P S ( ) Sotomayor: 0.18 Washington: 0.11 supreme: 0.10
41 KLSum: Extraction P C ( ) S = min KL(P CP S ) S:words(S) L P S ( ) Sotomayor: 0.15 Washington: 0.13 supreme: 0.12 Sotomayor: 0.18 Washington: 0.11 supreme: 0.10 See Paper For Details
42 KLSum: Performance SumBasic 5.3 KLSum
43 Improving Representation
44 Improving Representation Flexible stop words
45 Improving Representation Flexible stop words e.g. stock in financial document sets
46 Improving Representation Flexible stop words e.g. stock in financial document sets No pref. for multi-document usage
47 Improving Representation Flexible stop words e.g. stock in financial document sets No pref. for multi-document usage Many docs indicate content importance
48 Improving Representation Flexible stop words e.g. stock in financial document sets No pref. for multi-document usage Many docs indicate content importance Prefer words in early sentences
49 Adding Topics Background Distribution
50 Adding Topics Background Distribution φ B the: 0.12 of: washington: 0.04 policy: 0.03
51 Adding Topics Content Distribution
52 Adding Topics Content Distribution φ C Sotomayor: 0.16 supreme: 0.13 Obama: 12 court: 11 nominee: 10...
53 Adding Topics Document-Specific Distribution similar to [Daume & Marcu, 2006]
54 Adding Topics Document-Specific Distribution φ D Rush: 0.12 radical: 0.11 right: 0.09 court: 11 nominee: 10 similar to [Daume & Marcu, 2006]
55 Adding Topics Document-Specific Distribution φ D φ D Rush: 0.12 radical: 0.11 right: 0.09 court: 11 nominee: 10 Hutchinson: 0.14 past: 0.11 comments: 0.09 circuit: 11 statement: 10 similar to [Daume & Marcu, 2006]
56 Adding Topics Document-Specific Distribution φ D φ D Rush: 0.12 radical: 0.11 right: 0.09 court: 11 nominee: 10 Hutchinson: 0.14 past: 0.11 comments: 0.09 circuit: 11 statement: 10 φ D media: 0.12 believe: 0.11 rights: 0.09 cover: 0.11 fox: 0.10 similar to [Daume & Marcu, 2006]
57 Adding Topics Document-Specific Distribution φ D φ D Rush: 0.12 radical: 0.11 right: 0.09 court: 11 nominee: 10 Hutchinson: 0.14 past: 0.11 comments: 0.09 circuit: 11 statement: 10 φ D φ D media: 0.12 believe: 0.11 rights: 0.09 cover: 0.11 fox: 0.10 race: 0.13 Berkeley: 0.11 firemen: 0.09 panel: 11 decisions: 10 similar to [Daume & Marcu, 2006]
58 Adding Topics Document Level....
59 Adding Topics Document Level B D C. ψ t...
60 Adding Topics Document Level B D C. ψ t.. B D C. ψ t
61 Adding Topics Sentence B D C ψ t
62 Adding Topics Sentence B D C Word ψ t W
63 Adding Topics Sentence B D C Word Z ψ t W
64 Adding Topics Sentence B D C Word {B,D,C} Z ψ t W
65 Adding Topics Sentence B D C Word Z ψ t W
66 Adding Topics Sentence B D C Word Z ψ t W φ B φ D φ C
67 Adding Topics Representation Extraction φ C Sotomayor: 0.16 supreme: 0.13 Obama: 0.12 court: 0.11 nominee:
68 Adding Topics Representation Extraction φ C Sotomayor: 0.16 supreme: 0.13 Obama: 0.12 court: 0.11 nominee:
69 Performance SumBasic 5.3 KLSum 6.0 +Topic
70 Adding Bigrams Each sentence is a bag of bigrams
71 Adding Bigrams Each sentence is a bag of bigrams φ C Obama announced:0.09 Sonia Sotomayor: 0.08 Supreme Court:
72 Performance SumBasic 5.3 KLSum 6.0 +Topic 6.3 +Bigram
73 Structured Content Models
74 Structured Content Models General Content Sotomayor: 0.16 supreme: 0.13 court:
75 Structured Content Models General Content Sotomayor: 0.16 supreme: 0.13 court: Specific Content Sub-Stories
76 Structured Content Models General Content Sotomayor: 0.16 supreme: 0.13 court: Specific Content Sub-Stories born: 0.15 puerto: 0.13 mother: 0.12 father: confirmation: 0.16 Republican: 0.11 senators: 0.08 Limbaugh: race: 0.11 identity: 0.09 firefighters: 0.07 discrimination:
77 Example Sub-Story Biography Sonia Sotomayor Born to Puerto Rican parents who moved to the Bronx in New York during World War Two.
78 Example Sub-Story Biography Sonia Sotomayor Born to Puerto Rican parents born: 0.15 puerto: 0.13 mother: 0.12 who moved to the Bronx in New York during World War Two. father:
79 Example Sub-Story Confirmation Senate Republicans have combed over Sotomayor s record on the federal bench ahead of confirmation hearings.
80 Example Sub-Story Confirmation confirmation: 0.16 Republican: 0.11 senators: 0.08 Senate Republicans have combed over Sotomayor s record on the federal bench ahead of confirmation hearings. Limbaugh:
81 Structured Content Models φ C0 Sotomayor: 0.16 supreme: 0.13 Obama: φ C1 φ C2 φ C3 born: 0.15 puerto: 0.13 mother: 0.12 father: confirmation: 0.16 Republican: 0.11 senators: 0.08 Limbaugh: race: 0.11 identity: 0.09 firefighters: 0.07 discrimination:
82 Structured Topics General or Specific?....
83 Structured Topics General or Specific? B D C.. ψ t..
84 Structured Topics General or Specific? B D C G S.... ψ t ψ G
85 Structured Topics General or Specific? B D C G S.... ψ t B D C ψ G ψ t
86 Structured Topics General or Specific? B D C G S.... ψ t B D C ψ G G S ψ t ψ G
87 Structured Topics What specific topic?
88 Structured Topics What specific topic? Z S
89 Structured Topics What specific topic? Z S {1,2,3}
90 Structured Topics What specific topic? Z S 1
91 Structured Topics What specific topic? Z S 1 Sticky Specific Topics P (Z S Z S )= σ (1 σ)θ Z S Z S o.w. if Z S = Z S
92 Structured Topics What specific topic? Z S 1 Sticky Specific Topics P (Z S Z S )= σ (1 σ)θ Z S Z S o.w. if Z S = Z S
93 Structured Topics G S Sentence Z S B D C Word ψ G Z ψ t φ B φ D φ C0 W φ C1 φ C2 φ C3
94 Structured Topics Representation Extraction φ C0 Sotomayor: 0.16 supreme: 0.13 Obama: 0.12 court: 0.11 nominee:
95 Structured Topics Representation Extraction φ C0 Sotomayor: 0.16 supreme: 0.13 Obama: 0.12 court: 0.11 nominee:
96 Performance SumBasic 5.3 +Topic 6.3 +Struct 6.4 +Bigram
97 HierSum: Manual Evaluation Pairwise Comparison 23 participants, 69 judgments Each participant sees reference summary and two model summaries Pythy: State-of-the-art discriminative system. Highest automatic score and highperforming on manual content evaluation
98 Manual Evaluation Question Pythy Ours Overall Redundancy Coherence Focus 28 41
99 DUC 07 Results SumBasic 5.9 Pythy 8.7 Ours
100 Example General Summary Former House Speaker Newt Gingrich is asking a judge to force his estranged wife to turn over money he says she is hoarding. On Thursday, accusations of wrongdoing and the mining of dirt in the former U.S. House speaker's divorce case gave way to a secret settlement between Gingrich and his wife of 18 years, Marianne Gingrich. Gingrich filed for divorce July 29 amid allegations he is having an affair with 33-year-old congressional aide Callista Bisek. Newt Gingrich's attorney, Randy Evans, says Marianne Gingrich has refused to even discuss a settlement until she questions Bisek.
101 Example Topical Summary Callista Bisek Marianne Gingrich also wants to depose Callista Bisek, a congressional aide with whom the former U.S. House speaker has a relationship. And in motions filed last week in Superior Court for the District of Columbia, Callista Bisek, a clerk for the House Agriculture Committee, asked a judge to overturn a Georgia court order requiring her to answer questions about her relationship with Gingrich. Mayoue has said he intends to question Bisek about all aspects of her relationship with Newt Gingrich.
102 Example Topical Summary Gingrich Bio / Post-Speaker Life Gingrich is best known leading the Republican Party's takeover of the House in During that so-called Republican Revolution, Gingrich emphasized that "family values" should be a core pillar in American society. Since resigning as speaker and from the congressional seat he held for 20 years, Gingrich has been making a living giving speeches, sitting on corporate boards, consulting and appearing as a political analyst on Fox News. U.S. Rep. J.D. Hayworth (R-Ariz.) argued that Gingrich's new job as a political commentator for Fox News makes it inappropriate to include him in political gatherings. "Time marches on. He's gone on to other pursuits," Hayworth said.
103 Conclusion KL objective for sentence extraction summarization Topic models can yield state-of-the-art automatic and manual summary eval Also structured content models for topical summarization
104 Conclusion Thanks! Questions? If not, I have more examples...
105 Topical Summarization
106 Topical Summarization
U.S. Plant Patents and the Imazio Decision
U.S. Plant Patents and the Imazio Decision Robert J. Jondle, Ph.D., Esq. Castle Rock, Colorado (303) 799-6444 rjondle@jondlelaw.com www.jondlelaw.com Overview of U.S. Protection Options 1. Plant Patents
The Crucible. Background
The Crucible Background The Crucible is... Puritanism + Witchcraft + McCarthyism + Arthur Miller The Salem Witch Trials of 1692 Because of the harsh conditions the Puritans endured, there was safety in
The Crucible. By Arthur Miller
The Crucible By Arthur Miller Salem, Massachusetts, 1692 Early in 1692, a small group of girls in Salem fell ill, falling victim to hallucinations and seizures. In extremely religious Puritan New England,
EDITORIAL NOTE: PERSONAL/COMMERCIAL DETAILS ONLY HAVE BEEN DELETED. IN THE DISTRICT COURT AT AUCKLAND CIV [2016] NZDC 12626
![EDITORIAL NOTE: PERSONAL/COMMERCIAL DETAILS ONLY HAVE BEEN DELETED. IN THE DISTRICT COURT AT AUCKLAND CIV [2016] NZDC 12626](/thumbs/76/73139010.jpg "EDITORIAL NOTE: PERSONAL/COMMERCIAL DETAILS ONLY HAVE BEEN DELETED. IN THE DISTRICT COURT AT AUCKLAND CIV [2016] NZDC 12626")
EDITORIAL NOTE: PERSONAL/COMMERCIAL DETAILS ONLY HAVE BEEN DELETED. IN THE DISTRICT COURT AT AUCKLAND BETWEEN AND CIV 2015-092-000479 [2016] NZDC 12626 MEDICAL ASSURANCE SOCIETY NEW ZEALAND LIMITED Plaintiff
MOTION REGARDING PARENTING TIME FOC 65
Mecosta County Friend the Court 400 Elm Street P.O. Box 508 Big Rapids, MI 49307 (231) 592-0115 USE THIS FORM IF: MOTION REGARDING PARENTING TIME FOC 65 You have a court order through the Mecosta County
1 Evaluation of SMT systems: BLEU
1 Evaluation of SMT systems: BLEU Idea: We want to define a repeatable evaluation method that uses: a gold standard of human generated reference translations a numerical translation closeness metric in
10-1 Sequences as Functions. Determine whether each sequence is arithmetic. Write yes or no. 1. 8, 2, 12, 22
Determine whether each sequence is arithmetic. Write yes or no. 1. 8, 2, 12, 22 Subtract each term from the term directly after it. The common difference is 10. 3. 1, 2, 4, 8, 16 Subtract each term from
The League of Nations
Name: Date: Class: Social Studies NTI Day 8 The League of Nations It was Wilson's hope that the final treaty, drafted by the victors, would be even-handed, but the passion and material sacrifice of more
Delta RV Eighth Grade Social Studies Revised-2010
Delta RV Eighth Grade Social Studies Revised-2010 Principles of Constitutional Democracy Content Standard 1: Knowledge of the principles expressed in documents shaping constitutional democracy in the United
Rubrics and Exemplars: Grade 8 Responses to Biography & Literature
Rubrics and Exemplars: Grade 8 Focus Definition: To select one specific moment or idea and to develop just this moment or idea in your writing. Student writes a minimum of 140 words AND almost all sentences
Presents Clever Alice From "The Fairy Book" by Miss Mulock - 1 -
Presents Clever Alice From "The Fairy Book" by Miss Mulock - 1 - nce upon a time there was a man who had a daughter, who was called Clever OAlice; and when she was grown up, her father said, We must see
E!!! E
E!!! E..A 2009 " A!!!" http://www.ethiomedia.com/adroit/eprp_tplf_and_the_future.pdf E A U E E A A E E E A E A A / E A - A E E / A (.A. ) A ( E ) A I A IA E A E/ E A E E I / A / E - - - I E A A E E E A
The units on the average rate of change in this situation are. change, and we would expect the graph to be. ab where a 0 and b 0.
Lesson 9: Exponential Functions Outline Objectives: I can analyze and interpret the behavior of exponential functions. I can solve exponential equations analytically and graphically. I can determine the
The Great Gatsby. SAT Prep Test Examples Jestice March 2017
The Great Gatsby SAT Prep Test Examples Jestice March 2017 Vocabulary Looking these up in the dictionary and using them in sentences will help you understand the right answers. Grammar and Writing Writing
GEOGRAPHIC INFORMATION SYSTEM ANALYST I GEOGRAPHIC INFORMATION SYSTEM ANALYST II
CITY OF ROSEVILLE GEOGRAPHIC INFORMATION SYSTEM ANALYST I GEOGRAPHIC INFORMATION SYSTEM ANALYST II DEFINITION To perform professional level work in Geographic Information Systems (GIS) management and analysis;
BIG IDEAS. Area of Learning: SOCIAL STUDIES Urban Studies Grade 12. Learning Standards. Curricular Competencies
Area of Learning: SOCIAL STUDIES Urban Studies Grade 12 BIG IDEAS Urbanization is a critical force that shapes both human life and the planet. The historical development of cities has been shaped by geographic,
Beliefs Mini-Story Text
Beliefs Mini-Story Text Hello, this is AJ Hoge, and this is the mini story for Beliefs. Are you feeling good? Good posture? Deep breathing? Big smile? Moving? I hope so. I m not going to start this story
1 Propositional Logic
CS 2800, Logic and Computation Propositional Logic Lectures Pete Manolios Version: 384 Spring 2011 1 Propositional Logic The study of logic was initiated by the ancient Greeks, who were concerned with
T L S H. Doug Johnson
Doug Johnson T L S H 2 THE LEGEND OF SLEEPY HOLLOW Todays story is called The Legend of Sleepy Hollow. It is about something strange that happed long ago in a valley called Sleepy Hollow. It was written
SPIRITUAL GIFTS. ( ) ( ) 1. Would you describe yourself as an effective public speaker?
SPIRITUAL GIFTS QUESTIONNAIRE: SPIRITUAL GIFTS ( ) ( ) 1. Would you describe yourself as an effective public speaker? ( ) ( ) 2. Do you find it easy and enjoyable to spend time in intense study and research
{ Jurafsky & Martin Ch. 6:! 6.6 incl.
N-grams Now Simple (Unsmoothed) N-grams Smoothing { Add-one Smoothing { Backo { Deleted Interpolation Reading: { Jurafsky & Martin Ch. 6:! 6.6 incl. 1 Word-prediction Applications Augmentative Communication
I. How we believe II. Pattern recognition III. Polygraphs/memory IV. Astrology. 6-Nov-14 Phys 192 Lecture 9 1
I. How we believe II. Pattern recognition III. Polygraphs/memory IV. Astrology 6-Nov-14 Phys 192 Lecture 9 1 Question: How many Earth-like planets: Requirements: Good Places for Life Single star system
Lesson 4: Efficiently Adding Integers and Other Rational Numbers
Lesson 4: Efficiently Adding Integers and Other Rational Numbers Classwork Example 1: Rule for Adding Integers with Same Signs a. Represent the sum of 3 + 5 using arrows on the number line. i. How long
Case: 2:17-cv ALM-KAJ Doc #: 1 Filed: 10/26/17 Page: 1 of 10 PAGEID #: 1 UNITED STATES DISTRICT COURT SOUTHERN DISTRICT OF OHIO EASTERN DIVISION
Case 217-cv-00946-ALM-KAJ Doc # 1 Filed 10/26/17 Page 1 of 10 PAGEID # 1 UNITED STATES DISTRICT COURT SOUTHERN DISTRICT OF OHIO EASTERN DIVISION Madeleine Entine c/o Cooper & Elliott LLC 2175 Riverside
Language Models. CS6200: Information Retrieval. Slides by: Jesse Anderton
Language Models CS6200: Information Retrieval Slides by: Jesse Anderton What s wrong with VSMs? Vector Space Models work reasonably well, but have a few problems: They are based on bag-of-words, so they
Internet Appendix for Sentiment During Recessions
Internet Appendix for Sentiment During Recessions Diego García In this appendix, we provide additional results to supplement the evidence included in the published version of the paper. In Section 1, we
The Role of Pre-trial Settlement in International Trade Disputes (1)
183 The Role of Pre-trial Settlement in International Trade Disputes (1) Jee-Hyeong Park To analyze the role of pre-trial settlement in international trade dispute resolutions, this paper develops a simple
Miss Johnson s Plant Experiment
Miss Johnson s Plant Experiment Miss Johnson s Plant Experiment Miss Johnson, a second grade teacher, reached deep into her canvas bag and pulled out two plants. She placed the plants on a table at the
Evaluating Text Summarization Systems with a Fair Baseline from Multiple Reference Summaries
Evaluating Text Summarization Systems with a Fair Baseline from Multiple Reference Summaries Fahmida Hamid (B, David Haraburda, and Paul Tarau Department of Computer Science and Engineering, University
PENGUIN READERS. Five Famous Fairy Tales
PENGUIN READERS Five Famous Fairy Tales Introduction Jacob and Wilhelm Grimm the Brothers Grimm were good friends. Jacob was a quiet man and sometimes sad. Wilhelm was often very ill but he was a happier
Latent Variable Models in NLP
Latent Variable Models in NLP Aria Haghighi with Slav Petrov, John DeNero, and Dan Klein UC Berkeley, CS Division Latent Variable Models Latent Variable Models Latent Variable Models Observed Latent Variable
Trick or Treat UNIT 19 FICTION. #3893 Nonfiction & Fiction Paired Texts 100 Teacher Created Resources
FICTION Trick or Treat Hurry, children. It s almost time to go! Nancy took one last look in the mirror. She liked her costume. She was a cat for Halloween. Her shirt and pants were both black. Her mother
Punctuated Equilibrium and Institutional Friction in Comparative Perspective
Punctuated Equilibrium and Institutional Friction in Comparative Perspective Frank R. Baumgartner, Pennsylvania State University Christian Breunig, University of Washington Christoffer Green-Pedersen,
REVIEW OF THE EMERGING ISSUES TASK FORCE
REVIEW OF THE EMERGING ISSUES TASK FORCE Conclusions and Recommendations Financial Accounting Standards Board www.fasb.org REVIEW OF THE EMERGING ISSUES TASK FORCE BACKGROUND OF THE EITF The Financial
Grade 5 Lesson 14 Use the biography by Walter Dean Myers entitled James Forten on pages in your student reader to answer the questions below.
Name: Date: Grade 5 Lesson 14 Use the biography by Walter Dean Myers entitled James Forten on pages 421-431 in your student reader to answer the questions below. Directions Read the biography. Then answer
Generating Settlements. Through Lateral Thinking Techniques
Generating Settlements Through Lateral Thinking Techniques Vertical Thinking vs. Lateral Thinking Complementary Processes- VT is concerned with proving or developing concept patterns and LT is concerned
A fast and simple algorithm for training neural probabilistic language models
A fast and simple algorithm for training neural probabilistic language models Andriy Mnih Joint work with Yee Whye Teh Gatsby Computational Neuroscience Unit University College London 25 January 2013 1
Two-Variable Regression Model: The Problem of Estimation
Two-Variable Regression Model: The Problem of Estimation Introducing the Ordinary Least Squares Estimator Jamie Monogan University of Georgia Intermediate Political Methodology Jamie Monogan (UGA) Two-Variable
Chapter 19 Classwork Famous Scientist Biography Isaac...
Chapter 19 Classwork Famous Scientist Biography Isaac... Score: 1. is perhaps the greatest physicist who has ever lived. 1@1 2. He and are almost equally matched contenders for this title. 1@1 3. Each
Evaluation. Brian Thompson slides by Philipp Koehn. 25 September 2018
Evaluation Brian Thompson slides by Philipp Koehn 25 September 2018 Evaluation 1 How good is a given machine translation system? Hard problem, since many different translations acceptable semantic equivalence
Which pages are those of job ads?
Using Text as Data Which pages are those of job ads? Which pages are those of job ads? Topic classification There are a number of possible types of pages of advertisements: for jobs, for real estate, for
Saturday Science Lesson Plan Fall 2008
Saturday Science Lesson Plan Fall 2008 LEARNING OBJECTIVES STANDARDS 1.1.1 Observe, describe, draw, and sort objects carefully to learn about them. 1.2.6 Describe and compare objects in terms of number,
2-5 Dividing Integers
Find each quotient. 1. 40 ( 10) 2. 4 3 3. 26 ( 3) 4. 9 5. 48 3 6. 16 4 7. 36 ( 4) 8. 9 8 Evaluate each expression if s = 2 and t = 7. 9. 14s t 4 10. 35 11. 4t (2s) 7 12. Financial Literacy The following
coven Emily Lisa Benjamin High Noon Books Novato, CA
coven Emily Lisa Benjamin High Noon Books Novato, CA Series Editor: Holly Melton Editors: KC Moore, Steve Shea, Elly Rabben Designer: Deborah Anker Cover Design and Illustration: Kirk DouPonce/DogEared
Little Saddleslut (Greek version of Cinderella)
Little Saddleslut (Greek version of Cinderella) Edmund Martin Geldart Greek Easy 8 min read There were once three sisters spinning flax, and they said, Whosever spindle falls, let us kill her and eat her.
Student Activities. The Murder at the Vicarage. English Readers. Part 1 (Chapters 1 8) 4 Vocabulary Find seven words connected to the church.
Part 1 (Chapters 1 8) Are these sentences true or false? Rewrite any that you think are false. 1 Len sees Lawrence Redding and Mrs Protheroe kissing. 2 Miss Marple does not have a sense of humour. 3 Anne
THE FEDERAL DEATH PENALTY SYSTEM
U.S. Department of Justice Washington, DC THE FEDERAL DEATH PENALTY SYSTEM A STATISTICAL SURVEY (1988-2000) September 12, 2000 SURVEY OF THE FEDERAL DEATH PENALTY SYSTEM (1988-2000) U.S. Department of
Q1. Amina is making designs with two different shapes.
Q1. Amina is making designs with two different shapes. She gives each shape a value. Total value is 147 Total value is 111 Calculate the value of each shape. Q2. n stands for a whole number. 2n is greater
Notes on Mechanism Designy
Notes on Mechanism Designy ECON 20B - Game Theory Guillermo Ordoñez UCLA February 0, 2006 Mechanism Design. Informal discussion. Mechanisms are particular types of games of incomplete (or asymmetric) information
Math 6 Mid-Winter Recess
MOUNT VERNON CITY SCHOOL DISTRICT Children of Promise Math 6 Mid-Winter Recess Student Name: School Name: Teacher: Score: Module 1: Ratios and Unit Rates 1. Jasmine has taken an online boating safety course
IN THE SUPREME COURT OF BELIZE, A.D KIRK HALL BOWEN & BOWEN LTD.
IN THE SUPREME COURT OF BELIZE, A.D. 2006 CLAIM NO. 415 STACEY COLLINS CLAIMANT BETWEEN AND KIRK HALL BOWEN & BOWEN LTD. 1 st DEFENDANT 2 nd DEFENDANT Hearings 2009 28 th July 4 th September 18 th September
Exam III Review Math-132 (Sections 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, 8.1, 8.2, 8.3)
1 Exam III Review Math-132 (Sections 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, 8.1, 8.2, 8.3) On this exam, questions may come from any of the following topic areas: - Union and intersection of sets - Complement of
Read the selection and choose the best answer to each question. Then fill in the answer on your answer document. Lights Out
Read the selection and choose the best answer to each question. Then fill in the answer on your answer document. Characters DEXTER: Younger brother LEO: Older brother Lights Out 1 [Night. A boys bedroom
a n d n d n i n n d a o n e o t f o b g b f h n a f o r b c d e f g c a n h i j b i g y i t n d p n a p p s t e v i s w o m e b
Pre-Primer Word Search a n d n d n i n n d a o n e o t f o b g b f h n a f o r b c d e f g c a n h i j b i g y i t n d p n a p p s t e v i s w o m e b and big can for in is it me one 1 Pre-Primer Word
Organic Chemistry Syllabus
Organic Chemistry Syllabus 2017-2018 Instructor Information: Teacher: James Poindexter Room: Belle Vernon Area High School, Room 412 Contact: Phone: 724-808-2500; ext.2412 Email: james.poindexter@bellevernonarea.net
Physics 105 Spring 2017
Physics 105 Spring 2017 Lectures: Sec: 6: TR 5:30 6:45 pm in Moulton 208 Lab/discussion: Sec. 7: Mon 8:00-10:50 am in MLT 203 (Cory Russ) Sec. 8: Mon 11:00-1:50 pm in MLT 203 (Noah Osman) Sec. 9: Mon 3:00-5:50
CRF for human beings
CRF for human beings Arne Skjærholt LNS seminar CRF for human beings LNS seminar 1 / 29 Let G = (V, E) be a graph such that Y = (Y v ) v V, so that Y is indexed by the vertices of G. Then (X, Y) is a conditional
An extended summary of the NCGR/Berkeley Double-Blind Test of Astrology undertaken by Shawn Carlson and published in 1985
From: http://www.astrodivination.com/moa/ncgrberk.htm An extended summary of the NCGR/Berkeley Double-Blind Test of Astrology undertaken by Shawn Carlson and published in 1985 Introduction Under the heading
Homework 4, Part B: Structured perceptron
Homework 4, Part B: Structured perceptron CS 585, UMass Amherst, Fall 2016 Overview Due Friday, Oct 28. Get starter code/data from the course website s schedule page. You should submit a zipped directory
What is Text mining? To discover the useful patterns/contents from the large amount of data that can be structured or unstructured.
What is Text mining? To discover the useful patterns/contents from the large amount of data that can be structured or unstructured. Text mining What can be used for text mining?? Classification/categorization
HUMAN GEOGRAPHY AND CONTEMPORARY SOCIETY Global Patterns and Processes Spring 2009
HUMAN GEOGRAPHY AND CONTEMPORARY SOCIETY Global Patterns and Processes Spring 2009 Professor: Reece Jones Office: 412 Saunders Hall Email: reecej@hawaii.edu Office hours: M, T, W, Th 4:30 5:00 or by appointment
Grade Common Core Math
th 5 Grade Common Core Math Operations and Algebraic Thinking Printable Review Problems Standards Included: -Use parentheses, brackets, or braces in numerical expressions, and evaluate expressions with
Kesten C. Green Univ. of South Australia, Adelaide. March 23, Washington, D.C. 12 th International Conference on Climate Change
Are Forecasts of Dangerous Global Warming Scientific? Preparing for People vs. Alarmist Regulation J. Scott Armstrong The Wharton School, Univ. of Pennsylvania, Philadelphia Kesten C. Green Univ. of South
4. What does it mean once the letter "d" is formed when you draw a line on the moon?
Unit 5 Assessment Directions: Answer the questions; you may look back at the selection as often as needed. 1. Where would you most likely find this text? (a) Fairy Tales Book (b) Poetry Book (c) Diary
The Shunammite Woman s Land Restored 2 Kings 8:1-6
Lesson 111 The Shunammite Woman s Land Restored 2 Kings 8:1-6 MEMORY VERSE 2 KIN GS 8:6 Restore all that w as hers, and all the proc eeds of the field from the day that she left the land until now. WHAT
Categorization ANLP Lecture 10 Text Categorization with Naive Bayes
1 Categorization ANLP Lecture 10 Text Categorization with Naive Bayes Sharon Goldwater 6 October 2014 Important task for both humans and machines object identification face recognition spoken word recognition
ANLP Lecture 10 Text Categorization with Naive Bayes
ANLP Lecture 10 Text Categorization with Naive Bayes Sharon Goldwater 6 October 2014 Categorization Important task for both humans and machines 1 object identification face recognition spoken word recognition
A Discriminative Model for Semantics-to-String Translation
A Discriminative Model for Semantics-to-String Translation Aleš Tamchyna 1 and Chris Quirk 2 and Michel Galley 2 1 Charles University in Prague 2 Microsoft Research July 30, 2015 Tamchyna, Quirk, Galley
BIOGRAPHY OF MICHAEL FARADAY PART - 1. By SIDDHANT AGNIHOTRI B.Sc (Silver Medalist) M.Sc (Applied Physics) Facebook: sid_educationconnect
BIOGRAPHY OF MICHAEL FARADAY PART - 1 By SIDDHANT AGNIHOTRI B.Sc (Silver Medalist) M.Sc (Applied Physics) Facebook: sid_educationconnect WHAT WE WILL STUDY? CHILDHOOD STRUGGLE MAKING OF A GREAT SCIENTIST
Amy's Halloween Secret
Amy's Halloween Secret Amy's Halloween Secret by ReadWorks It was almost October 31, and Amy was excited. Halloween was her favorite day of the year. She found it more fun than Christmas, because she got
Calculator Exam 2009 University of Houston Math Contest. Name: School: There is no penalty for guessing.
Calculator Exam 2009 University of Houston Math Contest Name: School: Please read the questions carefully. Unless otherwise requested, round your answers to 8 decimal places. There is no penalty for guessing.
Language Modeling. Michael Collins, Columbia University
Language Modeling Michael Collins, Columbia University Overview The language modeling problem Trigram models Evaluating language models: perplexity Estimation techniques: Linear interpolation Discounting
TIPS PLANNING FORM FOR INFANTS AND TODDLERS
TIPS PLANNING FORM FOR INFANTS AND TODDLERS Tune In, Introduce the Book, Promote Language, Summarize the Book Book Title: Apples and Pumpkins Author: Anne Rockwell T: Tune In Engage the child/children
Physics 103 Astronomy Syllabus and Schedule Fall 2016
Physics 103 Astronomy Syllabus and Schedule Fall 2016 Instructor: April Hendley Phone: 453-2272 Office: Neckers 462 E-Mail: ahendley@siu.edu Office Hours: Tuesday: 10:00 am 12:00 noon Wednesday: 1:30 pm
A Probabilistic Model for Canonicalizing Named Entity Mentions. Dani Yogatama Yanchuan Sim Noah A. Smith
A Probabilistic Model for Canonicalizing Named Entity Mentions Dani Yogatama Yanchuan Sim Noah A. Smith Introduction Model Experiments Conclusions Outline Introduction Model Experiments Conclusions Outline
OREGON POPULATION FORECAST PROGRAM
OREGON POPULATION FORECAST PROGRAM PROGRAM OVERVIEW BACKGROUND Beginning in 1973 with the passage of Senate Bill (SB) 100, Oregon s growth management system has relied on population forecasts as the primary
N-grams. Motivation. Simple n-grams. Smoothing. Backoff. N-grams L545. Dept. of Linguistics, Indiana University Spring / 24
L545 Dept. of Linguistics, Indiana University Spring 2013 1 / 24 Morphosyntax We just finished talking about morphology (cf. words) And pretty soon we re going to discuss syntax (cf. sentences) In between,
Applied Natural Language Processing
Applied Natural Language Processing Info 256 Lecture 7: Testing (Feb 12, 2019) David Bamman, UC Berkeley Significance in NLP You develop a new method for text classification; is it better than what comes
GOVERNMENT MAPPING WORKSHOP RECOVER Edmonton s Urban Wellness Plan Mapping Workshop December 4, 2017
GOVERNMENT MAPPING WORKSHOP 12.4.17 RECOVER Edmonton s Urban Wellness Plan Mapping Workshop December 4, 2017 In July of 2017, City Council directed administration to develop RECOVER, Edmonton s Urban Wellness
Houston County School System Mathematics
Student Name: Teacher Name: Grade: 6th Unit #: 4b Unit Title: Analyzing Quantitative Relationships Approximate Start Date of Unit: January 4 Approximate End Date (and Test Date) of Unit: January 19 I can
Combining Difference-in-difference and Matching for Panel Data Analysis
Combining Difference-in-difference and Matching for Panel Data Analysis Weihua An Departments of Sociology and Statistics Indiana University July 28, 2016 1 / 15 Research Interests Network Analysis Social
Do you know the man that dropped out of school and still was one of the greatest physicists of the 20th century? That is Albert Einstein.
Do you know the man that dropped out of school and still was one of the greatest physicists of the 20th century? That is Albert Einstein. Albert was a man that thought for himself and changed the world
The Making Of A Great Contraction. With A Liquidity Trap And A Jobless Recovery. Columbia University
The Making Of A Great Contraction With A Liquidity Trap And A Jobless Recovery Stephanie Schmitt-Grohé Martín Uribe Columbia University November 5, 2013 A jobless recovery is a situation in which: Output
Chapter 9: Apportionment
Chapter 9: Apportionment Apportionment involves dividing something up, just like fair division. In fair division we are dividing objects among people while in apportionment we are dividing people among
4. The table shows the number of toll booths driven through compared to the cost of using a Toll Tag.
ALGEBRA 1 Fall 2016 Semester Exam Review Name 1. According to the data shown below, which would be the best prediction of the average cost of a -bedroom house in Georgetown in the year 2018? Year Average
Comparative Summarization via Latent Dirichlet Allocation
Comparative Summarization via Latent Dirichlet Allocation Michal Campr and Karel Jezek Department of Computer Science and Engineering, FAV, University of West Bohemia, 11 February 2013, 301 00, Plzen,
Explorers 4 Teacher s notes for the Comprehension Test: The Snow Queen
Explorers 4 Teacher s notes for the Comprehension Test: The Snow Queen Do this test after you have read the whole book with the class. Ask the children to fill in their name and the date at the top of
Do not copy, post, or distribute
14 CORRELATION ANALYSIS AND LINEAR REGRESSION Assessing the Covariability of Two Quantitative Properties 14.0 LEARNING OBJECTIVES In this chapter, we discuss two related techniques for assessing a possible
Appendix A: Topline questionnaire
Appendix A: Topline questionnaire 2016 RACIAL ATTITUDES IN AMERICA III FINAL TOPLINE FEBRUARY 29-MAY 8, 2016 NOTE: ALL NUMBERS ARE PERCENTAGES. THE PERCENTAGES LESS THAN 0.5% ARE REPLACED BY AN ASTERISK
Explorers 3 Teacher s notes for the Comprehension Test: The Magic Flute
Explorers 3 Teacher s notes for the Comprehension Test: The Magic Flute Do this test after you have read the whole book with the class. Ask the children to fill in their name and the date at the top of
Princes of Prophecy Alex s Program. by L. Ann Marie and some help from SOS
Princes of Prophecy Alex s Program by L. Ann Marie and some help from SOS This is a work of fiction. Names, characters, places, and incidents are either the product of the author s imagination or used
Name: Period: - Unit 2: Solving Equations. o One and two step equations: o Multi-step equations : o Multi-Step equations with fractions: /20
Targets 2.1: Students will be able to review one and two step equations 2.2: Students will be able to solve equations that require more than two steps. 2.3: Students will be able to solve equations that
AP Human Geography. Course Materials
AP Human Geography This is a syllabus for a two semester Advanced Placement Human Geography course that has been offered for several years at this school. The material covered is based on the AP Human
Writing: Looking Out the Window
ESOL Level Two Writing: This is an activity to use toward the end of the quarter to help students prepare for their writing assessment and to practice writing a simple narrative. It is also an opportunity
Annual PHA Plan. Effective July 1, 2017 June 30, Decatur Housing Authority
Decatur Housing Authority Annual PHA Plan Effective July 1, 2017 June 30, 2018 Decatur Housing Authority 7315 Hanna Street Fort Wayne, IN 46816 Phone: (260) 267-9300 Website: www.fwha.org PHA Plan Attachments
Statistical Methods for NLP
Statistical Methods for NLP Language Models, Graphical Models Sameer Maskey Week 13, April 13, 2010 Some slides provided by Stanley Chen and from Bishop Book Resources 1 Announcements Final Project Due,
Houston County School System Mathematics
Student Name: Teacher Name: Grade: 6th Unit #: 4b Unit Title: Analyzing Quantitative Relationships Approximate Start Date of Unit: Approximate End Date (and Test Date) of Unit: The following Statements
The Source of True Wisdom. Proverbs 1. Memory Verse: Proverbs 2:6
Chapter 2 The Source of True Wisdom Proverbs 1 Memory Verse: Proverbs 2:6 10 Vocabulary Quiz SAINTS Provide the dictionary definition. Proverbs 2:8 He guards the paths of justice, And preserves the way
STOCKHOLM UNIVERSITY Department of Economics Course name: Empirical Methods Course code: EC40 Examiner: Per Pettersson-Lidbom Number of creds: 7,5 creds Date of exam: Thursday, January 15, 009 Examination
Identify the scale of measurement most appropriate for each of the following variables. (Use A = nominal, B = ordinal, C = interval, D = ratio.
Answers to Items from Problem Set 1 Item 1 Identify the scale of measurement most appropriate for each of the following variables. (Use A = nominal, B = ordinal, C = interval, D = ratio.) a. response latency