INTRODUCTION TO MACHINE LEARNING FOR MEDICINE

Transcription

1 Fall 2017 INTRODUCTION TO MACHINE LEARNING FOR MEDICINE Carla E. Brdley Prfessr & Dean Cllege f Cmputer and Infrmatin Science Nrtheastern University

2 WHAT IS MACHINE LEARNING/DATA MINING? Figure is frm Fayyad, Piatetsky-Shapir, Smyth, and Uthurusamy. Advances in Knwledge Discvery and Data Mining, 1996; image fund at: www2.cs.uregina.ca/~dbd/cs831/ntes/kdd/kdd.gif

3 Fall 2017 SUPERVISED LEARNING NORTHEASTERN UNIVERSITY COLLEGE OF COMPUTER AND INFORMATION SCIENCE 3

4 SUPERVISED LEARNING Given: eample < 1, 2, nn, ff( 1, 2, nn ) > fr sme unknwn functin ff Find: A gd apprimatin t ff Gal: Apply ff t previusly unseen data Eample Applicatins: Regressin: ff is a cntinuus variable (e.g., predicting EDSS fr MS patients) Classificatin: ff is a discrete variable (e.g., predicting whether a patient has unilateral r bilateral Meniere s)

5 CLASSIFICATION EXAMPLE: CITATION SCREENING FOR SYSTEMATIC REVIEWS Systematic review: an ehaustive assessment f all the published medical evidence regarding a precise clinical questin e.g., Is aspirin better than leeches in inducing mre than 50% relief in patients with tensin headaches? Must find all relevant studies

6 TYPICAL WORKFLOW 26M PubMed SEARCH 10,000 Ptentially eligible SCREEN 500 Relevant

7 CITATION SCREENING Dctrs read these. They d rather be ding smething else.

8 GENERATING TRAINING DATA FOR SUPERVISED LEARNING Epert labels randm subset Induce (train) a classifier C ver Apply C t unlabeled eamples -

9 A DETOUR INTO TEXT ENCODING Classificatin algrithms perate n vectrs Feature space: an n-dimensinal representatin A bag-f-wrds eample: S 1 = Bstn drivers are frequently aggressive S 2 = The Bstn Red S frequently hit line drives

10 TEXT ENCODING: STOP WORDS S 1 = Bstn drivers are frequently aggressive S 2 = The Bstn Red S frequently hit line drives

11 TEXT ENCODING: LOWERCASING S 1 = bstn drivers are frequently aggressive S 2 = The bstn red s frequently hit line drives

12 TEXT ENCODING: STEMMING S 1 = bstn drive are frequent aggressive S 2 = The bstn red s frequent hit line drive

13 TEXT ENCODING: VOILA hit red s line bstn frequent drive aggressive S 1 = S 2 = A new sentence, S 3, cmes alng: I hate the red s. Which sentence is it mst similar t? S 3 =

14 SUPPORT VECTOR MACHINES: A HAND-WAVING EXPLANATION margin supprt vectrs Minimize: ww. ww

15 SUPPORT VECTOR MACHINES: THE NON-LINEARLY SEPARABLE CASE ε 6 ε 2 ε 11 Minimize: ww. ww + CC εε RR kk=11 kk

16 SUPERVISED LEARNING Epert labels randm subset Induce (train) a classifier C ver Apply C t unlabeled eamples -

17 SUPERVISED LEARNING What if we are clever in what eamples we label? Induce (train) a classifier C ver Apply C t unlabeled eamples -

18 ACTIVE LEARNING Key idea: have the epert label eamples mst likely t be helpful in inducing a classifier Need fewer labels fr gd classificatin perfrmance = less time/wrk/mney Need a scring functin ff: epected value f labeling Mst ppular strategy: uncertainty sampling

19 UNCERTAINTY SAMPLING (W/ SVMS) Which eamples shuld we label net?

20 UNCERTAINTY SAMPLING (W/ SVMS) Uncertainty sampling: label the eamples nearest the separating plane

21 UNCERTAINTY SAMPLING (W/ SVMS)

22 WHY OFF-THE-SHELF AL DOESN T WORK FOR CITATION SCREENING Imbalanced data; relevant class is very small (~5%), but sensitivity t this class is paramunt Randm Active (uncertainty) Recall Accuracy

23 WHY MIGHT UNCERTAINTY SAMPLING FAIL? Randm sampling Uncertainty sampling Hasty generalizatin: uncertainty sampling may miss clusters Pre-clustering desn t help unreliable in high-dimensins small clusters f interest

24 GUIDING AL WITH DOMAIN KNOWLEDGE Labeled terms: terms r n-grams whse presence is indicative f class membership tensin headache, leeches, aspirin migraine headache, mice Is aspirin better than leeches in inducing mre than 50% relief in patients with tensin headaches?

25 CO-TESTING FRAMEWORK (MUSLEA ET AL., 2000) Mdel 1 Mdel 2 F 1 () F 2 () If mdel 1 disagrees with mdel 2 abut, then is a gd pint t label

26 LABELED TERMS + CO-TESTING Mdel 1: Standard BOW (linear kernel) SVM Mdel 2: Rati f #ps terms t #neg terms Query strategy: Find all dcuments abut which the mdels disagree Select fr labeling items f maimum disagreement

27 COPD: GENETIC ASSOCIATIONS WITH COPD

28 MOST IMPORTANT REQUIREMENT FOR MACHINE LEARNING TO WORK: THE DATA Are the features predictive f the class? Hw nisy is the data? (attribute nise vs. class nise) D yu have enugh (labeled) data? Are the training samples representative? NORTHEASTERN UNIVERSITY COLLEGE OF COMPUTER AND INFORMATION SCIENCE 28

29 TRANSFER LEARNING A machine learning technique t imprve perfrmance leveraging n related knwledge A primary task n dataset TT An auiliary dataset TT aaaaaa TT and TT aaaaaa are usually related and have similar distributins Auiliary TT aaaaaa Primary TT

30 TRANSFER LEARNING EXAMPLES Predicting readmissin t hspitals Use data frm ther hspitals t predict fr yur hspital Predicting MS prgressin Cmbining data frm multiple physicians NORTHEASTERN UNIVERSITY COLLEGE OF COMPUTER AND INFORMATION SCIENCE 30

31 Fall 2017 UNSUPERVISED LEARNING NORTHEASTERN UNIVERSITY COLLEGE OF COMPUTER AND INFORMATION SCIENCE 31

32 CLUSTERING Given a set f data pints, each described by a set f attributes, find clusters such that: Inter-cluster similarity is maimized Intra-cluster similarity is minimized Requires the definitin f a similarity measure F1 F2

33 EXAMPLE: K-MEANS

34 EXAMPLE: K-MEANS

35 EXAMPLE: K-MEANS

36 EXAMPLE: K-MEANS

37 EXAMPLE: K-MEANS

38 EXAMPLE: K-MEANS

39 EXAMPLE: CONSTRAINED K-MEANS

40 EXAMPLE: CONSTRAINED K-MEANS

41 EXAMPLE: CONSTRAINED K-MEANS

42 CHALLENGES IN CLUSTERING MEDICAL DATA Cnfunding factr: One r a set f features whse effect will lead t undesirable clustering slutin if nt remved Clustering clinical data: Physician subjectivity Age fr neurlgical test scring in MS

43 EXAMPLE: VESTIBULAR DISORDERS Balance Functin Age

44 CLUSTERING WITH K = 2 Balance Functin Age

45 PROPOSED SOLUTION Remve the impact f cnfunding factr F via cnstraint-based clustering: 1. Bin the data int hmgeneus grups w.r.t. F 2. Apply clustering t each grup and generate pair-wise instance cnstraints 3. Apply cnstraint-based clustering t entire data

46 STEP 1: BINNING (STRATIFICATION) Categrical F: Create ne bin per categry Eample: ne bin per physician fr MS data Numeric F: Create bins f: Unifrm ranges r unifrm bin sizes Dmain knwledge Mre sphisticated binning methds, such as nnparametric density estimatin, etc

47 STEP 1: BINNING Balance Functin 40 Age

48 STEP 2: CLUSTER IN EACH BIN AND GENERATE CONSTRAINTS In each bin: Apply clustering (e.g., EM ver a miture f Gaussians) Number f clusters can be specified by dmain knwledge r inferred using criteria such as BIC Generate must-nt-link cnstraints fr pairs f instances in different clusters

49 STEP 2: CLUSTER EACH BIN Balance Functin 40 Age

50 STEP 2: GENERATE CONSTRAINTS Balance Functin 40 Age

51 STEP 3: APPLY CONSTRAINT BASED CLUSTERING TO THE ENTIRE DATA Balance Functin Age

52 Fall 2017 ANOMALY DETECTION NORTHEASTERN UNIVERSITY COLLEGE OF COMPUTER AND INFORMATION SCIENCE 52

53 ANOMALY DETECTION Given a set f data pints, each described by a set f attributes, pints that are far away frm mst f the ther pints als called utliers Requires the definitin f a similarity measure F1 F2

54 TYPES OF ANOMALY DETECTION Supervised Labelled nrmal and anmalus data Similar t rare (minrity) class mining Semi-supervised Labels available nly fr nrmal data Unsupervised N labelled data Assumptin: anmalies are rare cmpared t nrmal data

55 COMPLEXITIES OF ANOMALY DETECTION Where des the nrmal data cme frm? Feature selectin Metric Different parts f the space may have different densities NORTHEASTERN UNIVERSITY COLLEGE OF COMPUTER AND INFORMATION SCIENCE 55

56 COMPLEXITIES OF ANOMALY DETECTION Which f P1, P2 and P3 are anmalies? Distance frm p 3 t nearest neighbr p 3 Distance frm p 2 t nearest neighbr p 2 p 1

57 ANOMALY DETECTION EXAMPLE: DETECTING CORTICAL LESIONS 50 millin affected by epilepsy wrldwide One-third remain refractry t treatment One f the mst cmmn causes f TRE: Fcal Crtical Dysplasia (FCD) Treatment: Surgical resectin f the abnrmal crtical tissue (aka lesin) Visual MRI Eam Lesin Identificatin & Tracing Inter-Cranial EEG Analysis Resective Surgery 70-80% f histlgically verified FCD cases have nrmal MRI Chances f being seizure free after surgery: MRI-Psitive: 66% MRI-Negative: 29%

58 MACHINE LEARNING CHALLENGES Input data Surfaces f FCD patients (MRI) Resected tissue (MRI-Negatives): histpathlgically verified Generus margins t ensure cmplete lesin remval Eact lcatin f the lesin is unknwn Labels Resectin znes fr MRI-negatives Lesin tracings by neurradilgists fr MRI-psitives False psitives in training data False negatives in training data frm lng untreated epilepsy, trauma, etc.

59 PROPOSED SOLUTION Hierarchical Cnditinal Randm Fields fr Outlier Detectin Discard piel-level labels and use nly image-level labels Redefine FCD lesin as: a crtical regin which is an utlier when cmpared t the same regin acrss a ppulatin f nrmal cntrls Hierarchical Cnditinal Randm Field

60 RESULTS Tested n fifteen MRI-negative patients with successful surgery High detectin rate (80%) fr MRInegative patients with higher average recall and precisin

61 MY LAST WORDS There are many, many different learning algrithms, but the key t success is in having the right training data. MLHC is a great cnference. NORTHEASTERN UNIVERSITY COLLEGE OF COMPUTER AND INFORMATION SCIENCE 61