Formal Ontology Construction from Texts

Transcription

1 Formal Ontology Construction from Texts Yue MA Theoretical Computer Science Faculty Informatics TU Dresden June 16, 2014

2 Outline 1 Introduction Ontology: definition and examples 2 Ontology Construction Learning Formal Definitions

3 Web Development Tendency [Nova Spivack 2008]

4 Semantically Annotated Documents

5 Query on Annotated Documents w.r.t. Ontology

6 Questions How to answer queries given semantic labels? How to obtain semantic labels? How to have the underlying semantic model (ontology)?

7 Questions How to answer queries given semantic labels? How to obtain semantic labels? How to have the underlying semantic model (ontology)?

8 Questions How to answer queries given semantic labels? How to obtain semantic labels? How to have the underlying semantic model (ontology)?

9 Questions How to answer queries given semantic labels? natural language level: semantic search formal level: ontology-based data access How to obtain semantic labels? How to have the underlying semantic model (ontology)?

10 Questions How to answer queries given semantic labels? natural language level: semantic search formal level: ontology-based data access How to obtain semantic labels? ontology-based semantic annotation of texts How to have the underlying semantic model (ontology)?

11 Questions How to answer queries given semantic labels? natural language level: semantic search formal level: ontology-based data access How to obtain semantic labels? ontology-based semantic annotation of texts How to have the underlying semantic model (ontology)? Ontology construction (from texts and/or semi-structured data)

12 Outline 1 Introduction Ontology: definition and examples 2 Ontology Construction Learning Formal Definitions

13

14

15

16

17

18

19

20

21

22

23

24

25

26 Biomedical Ontologies: SNOMED, Gene Ontology, MeSH

27 Biomedical Ontologies: SNOMED, Gene Ontology, MeSH Not only a hierarchy!

28 SNOMED A medical ontology Computer processable medical terms (diseases, findings, microorganisms, etc.) Consistent way to index and aggregate clinical data Electronic health records to improve patient care A formal ontology In DL (Description Logic): EL Improve modeling by reasoning (e.g. finding errors) Semantic query answering (by exploiting deeper logical consequences)

29 How SNOMED Looks Like for Medical People Current Concept: Fully Specified Name: Baritosis (disorder) ConceptId: Defining Relationships: Is a Causative agent Associated morphology Finding site Associated with Metal pneumoconiosis Occupational lung disease Barium dust Barium Deposition of foreign material Deposition Lung structure (body structure) Barium dust This concept is fully defined. Descriptions (Synonyms): Synonym: Barium lung [ ]

30 Formal Definition and Its Generation from Text Baritosis Finding_site. Lung_structure

31 Formal Definition and Its Generation from Text Baritosis Finding_site. Lung_structure A r.b x A(x) y r(x, y) B(y)

32 Formal Definition and Its Generation from Text Baritosis Causative_agent. Barium_dust Finding_site. Lung_structure

33 Formal Definition and Its Generation from Text Baritosis Associated_morphology. Deposition_of _foreign_material Causative_agent. Barium_dust Finding_site. Lung_structure

34 Formal Definition and Its Generation from Text Baritosis Occupational_lung_disease Associated_morphology. Deposition_of _foreign_material Causative_agent. Barium_dust Finding_site. Lung_structure

35 Formal Definition and Its Generation from Text Baritosis Metal_pneumoconiosis Occupational_lung_disease Associated_morphology. Deposition_of _foreign_material Causative_agent. Barium_dust Finding_site. Lung_structure

36 Formal Definition and Its Generation from Text Baritosis Metal_pneumoconiosis Occupational_lung_disease Associated_morphology. Deposition_of _foreign_material Causative_agent. Barium_dust Finding_site. Lung_structure

37 Formal Definition and Its Generation from Text Baritosis Causative_agent. Barium_dust A r.b

38 Formal Definition and Its Generation from Text Baritosis Causative_agent. Barium_dust A r.b A B Baritosis is... caused by long-term exposure to barium dust.

39 Formal Definition and Its Generation from Text [Ma and Distel 13a, 13b, 13c; Tsatsaronis et al. 13; Petrova et al. 14] Baritosis Causative_agent. Barium_dust A r.b A? r (r {r 1, r 2,...r n }) B Baritosis is... caused by long-term exposure to barium dust.

40 Formal Definition and Its Generation from Text [Ma and Distel 13a, 13b, 13c; Tsatsaronis et al. 13; Petrova et al. 14] Baritosis Causative_agent. Barium_dust A r.b A? r (r {r 1, r 2,...r n }) B Baritosis is... caused by long-term exposure to barium dust. Baritosis Causative_agent.Barium_dust

41 More Sentences for Extracting Formal Definitions Baritosis is a benign type of pneumoconiosis, which is caused by long-term exposure to barium dust. Baritosis are nonfibrotic forms of pneumoconiosis that result from inhalation of iron oxide. Baritosis is one of the benign pneumoconiosis in which inhaled particulate matter lies in the lungs for years. Baritosis is due to inorganic dust lies in the lungs.

42 More Sentences for Extracting Formal Definitions Baritosis is a benign type of pneumoconiosis, which is caused by long-term exposure to barium dust. Baritosis are nonfibrotic forms of pneumoconiosis that result from inhalation of iron oxide. Baritosis is one of the benign pneumoconiosis in which inhaled particulate matter lies in the lungs for years. Baritosis is due to inorganic dust lies in the lungs. Exercises: try to write down the formal definitions extractable from these sentences

43 More Sentences for Extracting Formal Definitions Baritosis is a benign type of pneumoconiosis, which is caused by long-term exposure to barium dust. Baritosis are nonfibrotic forms of pneumoconiosis that result from inhalation of iron oxide. Baritosis is one of the benign pneumoconiosis in which inhaled particulate matter lies in the lungs for years. Baritosis is due to inorganic dust lies in the lungs. Exercises: try to write down the formal definitions extractable from these sentences Baritosis is... caused by long-term exposure to barium dust. Baritosis Causative_agent.Barium_dust

44 Table : Statistics of relationships about diseases in SNOMED #IsA #(NonIsA) #(NonIsA and Only3Relations) Total 17, , , , 395

45 Outline 1 Introduction Ontology: definition and examples 2 Ontology Construction Learning Formal Definitions

46 Outline 1 Introduction Ontology: definition and examples 2 Ontology Construction Learning Formal Definitions

47 Ontology Construction Manual, hence costly 830 terms (avg.) = 5.3 person months [Bontas, ISWC06] SNOMED 311, 000 concepts = 165 person years Automatic, from textual data Existing (un)structured data + New publications Formal definitions of new concepts

48 Existing Approaches Proposed: linguistic pattern based [Völker, 2009] Reviewer: a person who reviews a paper submitted to a conference Person reviews_a_paper_submitted_to_a_conference Person Reviews. a_paper_submitted_to_a_conference Person Reviews. (Paper Submitted_to. Conference)

49 Existing Approaches Proposed: linguistic pattern based [Völker, 2009] Reviewer: a person who reviews a paper submitted to a conference Person reviews_a_paper_submitted_to_a_conference Person Reviews. a_paper_submitted_to_a_conference Person Reviews. (Paper Submitted_to. Conference)

50 Existing Approaches Proposed: linguistic pattern based [Völker, 2009] Reviewer: a person who reviews a paper submitted to a conference Person reviews_a_paper_submitted_to_a_conference Person Reviews. a_paper_submitted_to_a_conference Person Reviews. (Paper Submitted_to. Conference)

51 Existing Approaches Proposed: linguistic pattern based [Völker, 2009] Reviewer: a person who reviews a paper submitted to a conference Person reviews_a_paper_submitted_to_a_conference Person Reviews. a_paper_submitted_to_a_conference Person Reviews. (Paper Submitted_to. Conference)

52 Existing Approaches Problems: Linguistic pattern: word variant problem? e.g. caused by, due to, result from,... Caused_by, Due_to, Result_from,... Causative_agent (NO!) (YES!) Dictionary: exhaustive, context-based disambiguation?

53 Our Approach Observations of SNOMED: Role set is relatively stable New concepts are increasing 61 roles, > 311, 000 concepts, > 1, 360, 000 relationships

54 Our Approach AM: Associated_morphology; CA: Causative_agent; FS: Finding_site Role Relationship example DL format AM (Parulis, Abscess) Parulis AM.Abscess CA (Rabies, Rabies_virus) Rabies CA.Rabies_virus FS (Headache, Head_structure) Headache FS.Head_structure

55 Our Approach AM: Associated_morphology; CA: Causative_agent; FS: Finding_site Role Relationship example DL format AM (Parulis, Abscess) Parulis AM.Abscess CA (Rabies, Rabies_virus) Rabies CA.Rabies_virus FS (Headache, Head_structure) Headache FS.Head_structure Intuition of our approach: 1 Learn a probabilistic model for SNOMED roles 2 Predict relationships for new concepts as definition candidates 3 Build EL definitions by selecting reliable candidates

56 Our Approach Relation extraction based approach: A machine learning approach Supervision by a knowledge base Input 1: large textual data (e.g. wikipedia articles, textbooks, PubMed abstracts) Input 2: large number of SNOMED relationships (e.g. (Rabies, Rabies_virus): Causative_agent ) Output: relationship suggestions for concepts to be defined (C tbd, A): r of weight w C tbd r.a of weight w SNOMED Supervision for EL Definition Extraction

57 SNOMED Supervision for EL Definition Extraction Producing definition candidates in the Snomed style : Neither predefined dictionary nor linguistic patterns required Large amount of existing SNOMED definitions and medical texts as training data Robust for contextual disambiguation Notations: R: SNOMED relationships D train : textual documents for training D test : textual documents for test

58 Architecture 1: Data Preparation Training data extraction: Alignment of D train with R : Annotating D train with SNOMED concepts (by Metamap, U.S. National Library of Medicine) Dengue virus is the causative agent of both classical dengue fever and the more severe dengue hemorrhage fever and dengue shock syndrome. (Dengue_virus_(organism), Dengue_(disorder)): Causative_agent Features extraction from training data: between-words, left-words, right-words, POS, syntactic dependency of words,... left-words between-words right-words is the causative agent of both and the more

59 Architecture 1: Data Preparation Training data extraction: Alignment of D train with R : Annotating D train with SNOMED concepts (by Metamap, U.S. National Library of Medicine) Dengue virus is the causative agent of both classical dengue fever and the more severe dengue hemorrhage fever and dengue shock syndrome. (Dengue_virus_(organism), Dengue_(disorder)): Causative_agent Features extraction from training data: between-words, left-words, right-words, POS, syntactic dependency of words,... left-words between-words right-words is the causative agent of both and the more

60 Architecture 2: Training Phase Classification with a multiclass classifier Given features extracted for the given N SNOMED roles (from data preparation phase) To learn highly weighted features for each role Causitive_agent (CA) Associated_morphology (AM) Finding_site (FS) caused, cause, from the, by a, agent of, an infection of displacement of, medical condition characterized of, in, affects only, infection of Table : Examples of highly weighted features for 3 roles

61 Architecture 3: Test Phase Annotating D test : Baritosis is a benign type of pneumoconiosis, which is caused by long-term exposure to barium dust. Baritosis_(disorder), Barium_dust_(substance) Feature extraction: left-words between-words right-words is a benign type of pneumoconiosis, which is caused by long-term exposure to Table : Test data for Baritosis left-words between-words right-words is the causative agent of both and the more is caused by two cestodes tapeworm species is the bacterium that causes whooping Table : Training data for Causative_agent

62 Architecture 3: Test Phase Annotating D test : Baritosis is a benign type of pneumoconiosis, which is caused by long-term exposure to barium dust. Baritosis_(disorder), Barium_dust_(substance) Feature extraction: left-words between-words right-words is a benign type of pneumoconiosis, which is caused by long-term exposure to Table : Test data for Baritosis left-words between-words right-words is the causative agent of both and the more is caused by two cestodes tapeworm species is the bacterium that causes whooping Table : Training data for Causative_agent

63 Architecture 3: Test Phase Annotating D test : Baritosis is a benign type of pneumoconiosis, which is caused by long-term exposure to barium dust. Baritosis_(disorder), Barium_dust_(substance) Feature extraction: left-words between-words right-words is a benign type of pneumoconiosis, which is caused by long-term exposure to Table : Test data for Baritosis left-words between-words right-words is the causative agent of both and the more is caused by two cestodes tapeworm species is the bacterium that causes whooping Table : Training data for Causative_agent

64 Preliminary Experiment: Settings Train textual sentences D train : from Web by Dog4dag (Biotec, Dresden) Classifier: Stanford maximum entropy classifier Feature: between words Concept to be defined: C tbd (e.g. Baritosis) R: relationships from SNOMED definitions excluding C tbd #D train #Sentences(AM) #Sentences(CA) #Sentences(FS) Table : Statics of training data (AM: Associated_morphology; CA: Causative_agent; FS: Finding_site)

65 Preliminary Experiment: Results Sent Concept Pairs Should-be Predicted Score S1 Baritosis, Deposition AM AM S2 Baritosis, Foreign_body AM AM S3 Baritosis, Barium_dust CA CA S4 Baritosis, Barium_dust CA CA S5 Baritosis, Barium_dust CA CA S6 Baritosis, Lung_structure FS FS S7 Baritosis, Lung_structure FS AM A r B = A r. B Baritosis Associated_morphology.Deposition Baritosis Associated_morphology.Foreign_body Baritosis Causative_agent.Barium_dust Baritosis Finding_site.Lung_structure Baritosis Associated_morphology.Lung_structure

66 Comparison of Learned and Given Definitions Baritosis Metal_pneumoconiosis Occupational_lung_disease Associated_morphology.Deposition Associated_morphology.Deposition_of _foreign_material Causative_agent.Barium Causative_agent.Barium_dust Finding_site.Lung_structure Associated_with.Barium Associated_with.Barium_dust Associated_morphology.Foreign_body (discovered from text) Causative_agent Associated_with

67 Comparison of Learned and Given Definitions Baritosis Metal_pneumoconiosis Occupational_lung_disease Associated_morphology.Deposition Associated_morphology.Deposition_of _foreign_material Causative_agent.Barium Causative_agent.Barium_dust Finding_site.Lung_structure Associated_with.Barium Associated_with.Barium_dust Associated_morphology.Foreign_body (discovered from text) Causative_agent Associated_with Barium_dust Barium

68 Formal Reasoning Brings More Training Data... Table : Sizes of the Explicit and Inferred Relationships for the relations: Associated_morphology, Causative_agent, and Finding_site Associated_ morphology Causative_ agent Finding_ site InfRB (after reasoning) 503, , 794 1, 306, 354 ExpRB (before reasoning) 32, , , 079

69 Formal Reasoning Brings More Training Data... [ Ma and Mendez 2013]

70 Formal Reasoning Brings More Training Data... [ Ma and Mendez 2013]

71 Detailed Experiments Modeling Corpora Relation Set Relationship Exp1 MeSH SNOMED InfRB Exp2 SemRep SemRep SemRep Exp3 WIKI+D4D SNOMED InfRB Feature Engineering Data Lexical Semantic Size Exp1 3-grams 424 Exp2 3-grams UMLS 1357 Exp3 3-grams SNOMED 9292 In all experiments Support Vector Machines are used for multi-class classification.

72 Detailed Experiments Modeling Corpora Relation Set Relationship Exp1 MeSH SNOMED InfRB Exp2 SemRep SemRep SemRep Exp3 WIKI+D4D SNOMED InfRB Feature Engineering Data Lexical Semantic Size Exp1 3-grams 424 Exp2 3-grams UMLS 1357 Exp3 3-grams SNOMED 9292 F-score without Types with Types Exp1 74% 99.1% Exp2 51% 54% 94% Exp3 58% 70% 100% In all experiments Support Vector Machines are used for multi-class classification.

73 Why Semantic Type Important? Causitive_agent caused, cause, from the, by a, agent of, an infection of Associated_morphology displacement of, medical condition characterized Finding_site of, in, affects only, infection of Example. an infection of virus : the existence of a causative agent relation infection of stomach : the location of a disease (i.e. its finding site)

74 Summary of the Approach 1 Learn highly weighted features for each role 2 Predict relationships for new concepts as def. candidates 3 Build EL definitions by selecting reliable candidates

75 Summary of the Approach 1 Learn highly weighted features for each role 2 Predict relationships for new concepts as def. candidates 3 Build EL definitions by selecting reliable candidates

76 Reliable Candidates: What and Why? Example Results by automatic learning [MaDistel13] Baritosis Causative_agent Barium_compound, Baritosis Causative_agent Dust, Baritosis Finding_site Lung_structure, Baritosis Causative_agent Barium_dust, Translated into DL: conjunction of candidates Baritosis Causative_agent.Barium_compound Causative_agent.Dust Finding_site.Lung_structure Causative_agent.Barium_dust Incorrect candidate according to SNOMED! (using SNOMED type as features doest not help)

77 Reliable Candidates: What and Why? Example Results by automatic learning [MaDistel13] Baritosis Causative_agent Barium_compound, Baritosis Causative_agent Dust, Baritosis Finding_site Lung_structure, Baritosis Causative_agent Barium_dust, Translated into DL: conjunction of candidates Baritosis Causative_agent.Barium_compound Causative_agent.Dust Finding_site.Lung_structure Causative_agent.Barium_dust Incorrect candidate according to SNOMED! (using SNOMED type as features doest not help)

78 Reliable Candidates: What and Why? Example Results by automatic learning [MaDistel13] Baritosis Causative_agent Barium_compound, Baritosis Causative_agent Dust, Baritosis Finding_site Lung_structure, Baritosis Causative_agent Barium_dust, Translated into DL: conjunction of candidates Baritosis Causative_agent.Barium_compound Causative_agent.Dust Finding_site.Lung_structure Causative_agent.Barium_dust Incorrect candidate according to SNOMED! (using SNOMED type as features doest not help)

79 Reliable Candidates: How? Logical Constraints: Intuition Baritosis Causative_agent. Barium_compound Permissible values for attributes [SNOMED User Guide]: Causative_agent: {Organism, Substance, Physical object, Physical force, Pharmaceutical/biologic product} Substance SNOMED CT Concept Chemical direct hierarchy indirect hierarchy Dust Barium_AND/OR_barium_compound Barium Barium_compound Barium_dust

80 Concept Adjustment for Candidate Selection Formal constrain formalization: D: a complex concept description; X: a concept variable Positive constrains: D X (1) X D (2) Negative constrains: D X (3) X D (4) Constrain Satisfiability For a concept description C not using any concept variable X, we say C satisfies Constraint 1 (resp. 2) w.r.t. a TBox T if T = D C (resp. T = C D). C satisfies Constraint 3 (resp. 4) w.r.t. a TBox T if T = D C (resp. T = C D).

81 Concept Adjustment for Candidate Selection Problem Formalization Input: a set of atomic candidate concept descriptions S, an ontology T and a set of constraints C of the forms 1 4. Problem (Concept Adjustment (CA)) Question: Is there a subset S of S such that S satisfies all the constraints in C? Problem (Maximal Confidence Concept Adjustment (MCA)) Additional input: a confidence function of candidates wt: C [0, 1]. Question: Is there a subset S of S such that S satisfies all the constraints in C and min{wt(s) S S } k?

82 Complexity Positive constrains: D X (1) X D (2) Negative constrains: D X (3) X D (4) Tractable variants (any combination without Type 4): For S S, if S satisfies a constraint X D or D X, so does S. Subsumption reasoning in EL is tractable Intractable variants (any combination containing Type 2+4): obviously, in NP NP-hard: reduction from SAT

83 SAT Encoding Type 1: dealt with by a P-time preprocessing step S 0 = {S S (D X) C: T = D S} C 0 = {c C c of type (2) or (3)}. (5) Type 2 4: f X D = T =S i D s i. f D X = s i, T =D S i f X D = s i. T =S i D D atom of D Theorem. S is a solution to the CA problem (T, S, C) iff φ S makes the formula f = c C f c true.

84 Construction of Formal Constraints Automatic: Original SNOMED, i.e. without the target concept removed, to simulate an expert If SNOMED = TargetConcept C then AutoConstraints.add(TargetConcept C) If SNOMED = C TargetConcept then AutoConstraints.add(C TargetConcept) Explicitly defined domain and range information for roles by SNOMED User Guide Forall R SNOMED If B Range(R) then AutoConstraints.add(TargetConcept R.B) If TargetConcept Domain(R) then AutoConstraints.add(TargetConcept R. )

85 Construction of Formal Constraints Automatic: Original SNOMED, i.e. without the target concept removed, to simulate an expert If SNOMED = TargetConcept C then AutoConstraints.add(TargetConcept C) If SNOMED = C TargetConcept then AutoConstraints.add(C TargetConcept) Explicitly defined domain and range information for roles by SNOMED User Guide Forall R SNOMED If B Range(R) then AutoConstraints.add(TargetConcept R.B) If TargetConcept Domain(R) then AutoConstraints.add(TargetConcept R. )

86 Construction of Formal Constraints Interactive: While MoreConstraints NewImplication new subsumptions entailed after adding candidates ForAll p in NewImplication question Is p is desired If yes then InteConstraints.add(p) Else InteConstraints.add( p) Constraints AutoConstraints InteConstraints Candidates CandidateSelection(Candidates, Constraints, portend)

87 Evaluation Experiment Set Up Target concepts: 853 SNOMED concepts (descendants of Disease, mentioned in our textual data) Texts: from WIKI and D4D (61,035 sentences, 1,084,924 words) Learning Definition Candidates from Texts [MaDistel13] One-concept-leave-out evaluation

88 Evaluation Metric Given a set of candidates Cands = {A R B : A, B are concept names and R is a role name}, the reasoning based precision can be defined as follows: Precision = {A R B Cands : SNOMED = A R B}. Cands

89 Results 276 concepts whose candidates generated by NLP having Precision < 1 are evaluated: 1.2 Unadjusted Candidates Adjusted Candidates 1 Precision Concept Figure : Precision with/without Concept Adjustment

90 Summary Baritosis Metal_pneumoconiosis Causative_agent.Barium_dust EL Finding_site.Lung_structure... Associated_morphology.Deposition_of _foreign_material

91 Summary Baritosis Metal_pneumoconiosis Causative_agent.Barium_dust EL Finding_site.Lung_structure... Associated_morphology.Deposition_of _foreign_material

92 Summary Baritosis Metal_pneumoconiosis Causative_agent.Barium_dust EL Finding_site.Lung_structure... Associated_morphology.Deposition_of _foreign_material Baritosis is a benign type of pneumoconiosis, which is caused by long-term exposure to barium dust. It is typically characterized by a mixture of very fine punctate and... in the lung.

93 Summary Baritosis Metal_pneumoconiosis EL Causative_agent.Barium_dust Finding_site.Lung_structure... Associated_morphology.Deposition_of _foreign_material Semantic Annotation Baritosis is a benign type of pneumoconiosis, which is caused by long-term exposure to barium dust. It is typically characterized by a mixture of very fine punctate and... in the lung.

94 Summary Baritosis Metal_pneumoconiosis EL Causative_agent.Barium_dust Finding_site.Lung_structure... Associated_morphology.Deposition_of _foreign_material Semantic Annotation Ontology Construction Baritosis is a benign type of pneumoconiosis, which is caused by long-term exposure to barium dust. It is typically characterized by a mixture of very fine punctate and... in the lung.