Application of Jaro-Winkler String Comparator in Enhancing Veterans Administrative Records
|
|
|
- Denis Osborne Burns
- 5 years ago
- Views:
Transcription
1 Application of Jaro-Winkler String Comparator in Enhancing Veterans Administrative Records Hyo Park, Eddie Thomas, Pheak Lim The Office of Data Governance and Analytics Department of Veterans Affairs FCSM, 2018 The findings and conclusions in this presentation are those of the authors and do necessarily represent the views of the Department of Veterans Affairs.
2 Introduction U.S. Veterans Eligibility Trends and Statistics (USVETS) - an integrated Veterans database using administrative records from internal and external data sources Social Security Number (SSN) was used as the primary unique identifier to link records across sources. Records linked by SSN alone resulted in matching records from different individuals. Utilized string comparator in conditional matching of multiple data sources to improve the data quality.
3 Potential Sources of Name Variation Incorrect Data Entry e.g. erroneously typed SSNs Misspelled names Given Names vs. Nick Names Maiden Names vs. Married Names Ethnic Names vs. English Names
4 SSA Verification File SSN, names, genders, and dates of birth sent to SSA for verification. SSA returns the input fields along with verification codes Produces the SSAV final dataset that contains SSA validated information, SSN, and best names Final dataset includes all necessary variables to validate the SSN for each source.
5 Record Matching without SSA Verification File Source John Smith James Jones Source John Smith James Jones USVETS John Smith John Smith James Jones
6 Using SSA Verification to select correct name Source John Smith James Jones John Smith String Comparator SSA Verification File John Smith James Jones Source John Smith James Jones John Smith USVETS John Smith James Jones
7 Cleaning Source Data Over 20 different source data ingested All source data are first cleaned and formatted Cleaning and Formatting Macro Programs: Remove invalid SSNs and invalid names (e.g., DEMO, DONOT) converts character dates to MM/DD/YYYY converts dollar fields to SAS DOLLAR formats Verifies SSN and name formats Create a suffix name field Clean date fields and convert to SAS date formats
8 Validation Process Verify each SSN and name combination If possible, verify gender and date of birth Create strings based on name and date-of-birth for source file and SSA Verification File Apply sting comparators to compare the two strings for each record
9 Datasets and Samples U.S. Veterans Eligibility Trends and Statistics (USVETS) 590,233 Unique Records from Chapter 33 Education Benefits (FY2016) Linked to SSA Verification File 13,145,484 Unique Records from Veterans Affairs/Department of Defense Identity Repository (VADIR) Active Component File Linked to SSA Verification File A random sample of 1,000 records for each source for classification analysis and ROC Curve Analysis, where string match scores are less than 1.
10 String Comparators Jaro Distance Jaro-Winkler Levenshtein Edit Distance
11 Jaro Distance d j = 1 3 ( m S 1 + m S 2 + m t m ) [1] S 1 and S 2 are lengths of S 1 and S 2 respectively; m is a number of matching characters; and t is a number of transpositions. Two characters are called matching if the one from the string S 1 agrees with another one from the string S 2 which is located not farther than [ max ( S 1, S 2 ) ]- 1 2
12 Jaro-Winkler d w = d j + (1 d j )( d j is a Jaro Distance m (p+1) S 1 + S 2 2(p 1) ) [1] p is the length of common prefix, up to 4 characters Adjust for similar characters, common prefix, and longer string
13 Levenshtein Edit Distance The minimum number of edit steps required to convert one string to the other edit steps include insertion, deletion, and substitution. x e = 1 e [1] n e is the edit of the two strings n is the maximum edit length between two strings
14 Examples S1=PHEAKDEYLIM, S2=PHEAKLIM Jaro Distance: S1 =11, S2 =8, m=8, t=0 d j = 1 3 ( ) = Jaro-Winkler: p = 4 8 (4+1) d w = ( )( ) = (4 1) Levenshtein Edit Distance: x e = =
15 Comparing Jaro-Winkler with Jaro Distance and Levenshtein Distance for Chapter 33
16 Comparing Jaro-Winkler with Jaro Distance and Levenshtein Distance for VADIR
17 Classification Table [2] Observed Outcome Expected Outcome P (Positive) N (Negative) Matched TP FP PP Unmatched FN TN PN OP ON TOT True Positives (TP) False Positives (FP) True Negatives (TN) False Negatives (FN) PP / PN OP / ON TOT the number of cases which were correctly classified to be positive the number of cases which were incorrectly classified as positive (Type I Error) the number of cases which were correctly classified to be negative the number of cases which were incorrectly classified as negative (Type II Error) Predictive Positive / Predictive Negative Observed Positive / Observed Negative Total Sample Size
18 Analysis of Classification Table - Chapter P N 0.85 P N 0.95 P N M (T) M M JW U (F) 9 5 Total U 5 9 Total U 0 14 Total Accuracy = Accuracy = Accuracy = Jaro 0.7 P N M U 9 5 Total Accuracy = P N M U 0 14 Total Accuracy = P N M U 0 14 Total Accuracy = LEV 0.7 P N M U P N M U P N M U 0 14 Total Total Total Accuracy = Accuracy = Accuracy = 0.379
19 Definition - Sensitivity, Specificity, and 1- Specificity Sensitivity = true positive rate = TP/(TP+FN) Specificity = true negative rate = TN/(TN+FP) 1- Specificity = false positive rate = FP/(FP+TN)
20 Sensitivity / Specificity Analysis Chapter 33 Cutoff Sensitivity (TPR) JW Jaro LEV 1-Specificity (FPR) Sensitivity 1-Specificity Sensitivity 1-Specificity
21 Analysis of Classification Table - VADIR JW 0.7 P N M U 8 2 Total P N M U 8 2 Total P N M U 2 8 Total Accuracy= Accuracy= Accuracy= P N M P N M P N M Jaro U 8 2 Total U 3 7 Total U 0 10 Total Accuracy= Accuracy= Accuracy= LEV 0.7 P N M U P N M U P N M U 0 10 Total Accuracy= Total Accuracy= Total Accuracy= 0.121
22 Sensitivity / Specificity Analysis VADIR JW Jaro LEV Cutoff Sensitivity 1-Specificity Sensitivity 1-Specificity Sensitivity 1-Specificity
23 Receiver Operating Characteristic (ROC) Curve
24 ROC Curve and Area Under Curve - Chapter 33 TYPE AUC JW Jaro LEV
25 ROC Curve and Area Under Curve - VADIR TYPE AUC JW Jaro LEV 0.776
26 Conclusion and Future Work Jaro-Winkler and Jaro Distance performed considerably better than Levenshtein Distance, especially at high cutoff points. String comparator can enhance the quality of identity matching over that of solely based on SSN. Explore name variations due to ethnic names Explore the selection of a threshold that will results in optimal identity matching quality
27 References [1] Yancey, W.E. (2005), Evaluating String Comparator Performance for Record Linkage, research report RRS 2005/05 at [2]
Performance Evaluation
Performance Evaluation Confusion Matrix: Detected Positive Negative Actual Positive A: True Positive B: False Negative Negative C: False Positive D: True Negative Recall or Sensitivity or True Positive
Large-scale Data Linkage from Multiple Sources: Methodology and Research Challenges
Large-scale Data Linkage from Multiple Sources: Methodology and Research Challenges John M. Abowd Associate Director for Research and Methodology and Chief Scientist, U.S. Census Bureau Based on NBER Summer
Performance Evaluation
Performance Evaluation David S. Rosenberg Bloomberg ML EDU October 26, 2017 David S. Rosenberg (Bloomberg ML EDU) October 26, 2017 1 / 36 Baseline Models David S. Rosenberg (Bloomberg ML EDU) October 26,
Evaluation & Credibility Issues
Evaluation & Credibility Issues What measure should we use? accuracy might not be enough. How reliable are the predicted results? How much should we believe in what was learned? Error on the training data
Performance evaluation of binary classifiers
Performance evaluation of binary classifiers Kevin P. Murphy Last updated October 10, 2007 1 ROC curves We frequently design systems to detect events of interest, such as diseases in patients, faces in
Model Accuracy Measures
Model Accuracy Measures Master in Bioinformatics UPF 2017-2018 Eduardo Eyras Computational Genomics Pompeu Fabra University - ICREA Barcelona, Spain Variables What we can measure (attributes) Hypotheses
Bayesian Estimation of Bipartite Matchings for Record Linkage
Bayesian Estimation of Bipartite Matchings for Record Linkage Mauricio Sadinle msadinle@stat.duke.edu Duke University Supported by NSF grants SES-11-30706 to Carnegie Mellon University and SES-11-31897
Lecture 4 Discriminant Analysis, k-nearest Neighbors
Lecture 4 Discriminant Analysis, k-nearest Neighbors Fredrik Lindsten Division of Systems and Control Department of Information Technology Uppsala University. Email: fredrik.lindsten@it.uu.se fredrik.lindsten@it.uu.se
Bayesian Decision Theory
Introduction to Pattern Recognition [ Part 4 ] Mahdi Vasighi Remarks It is quite common to assume that the data in each class are adequately described by a Gaussian distribution. Bayesian classifier is
Data Mining algorithms
Data Mining algorithms 2017-2018 spring 02.07-09.2018 Overview Classification vs. Regression Evaluation I Basics Bálint Daróczy daroczyb@ilab.sztaki.hu Basic reachability: MTA SZTAKI, Lágymányosi str.
Performance Evaluation and Comparison
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Cross Validation and Resampling 3 Interval Estimation
Applied Machine Learning Annalisa Marsico
Applied Machine Learning Annalisa Marsico OWL RNA Bionformatics group Max Planck Institute for Molecular Genetics Free University of Berlin 22 April, SoSe 2015 Goals Feature Selection rather than Feature
Computational paradigms for the measurement signals processing. Metodologies for the development of classification algorithms.
Computational paradigms for the measurement signals processing. Metodologies for the development of classification algorithms. January 5, 25 Outline Methodologies for the development of classification
Moving Average Rules to Find. Confusion Matrix. CC283 Intelligent Problem Solving 05/11/2010. Edward Tsang (all rights reserved) 1
Machine Learning Overview Supervised Learning Training esting Te Unseen data Data Observed x 1 x 2... x n 1.6 7.1... 2.7 1.4 6.8... 3.1 2.1 5.4... 2.8... Machine Learning Patterns y = f(x) Target y Buy
A Bayesian Partitioning Approach to Duplicate Detection and Record Linkage
A Bayesian Partitioning Approach to Duplicate Detection and Record Linkage Mauricio Sadinle msadinle@stat.duke.edu Duke University Supported by NSF grants SES-11-30706 to Carnegie Mellon University and
Optical Character Recognition of Jutakshars within Devanagari Script
Optical Character Recognition of Jutakshars within Devanagari Script Sheallika Singh Shreesh Ladha Supervised by : Dr. Harish Karnick, Dr. Amit Mitra UGP Presentation, 10 April 2016 OCR of Jutakshars within
Class 4: Classification. Quaid Morris February 11 th, 2011 ML4Bio
Class 4: Classification Quaid Morris February 11 th, 211 ML4Bio Overview Basic concepts in classification: overfitting, cross-validation, evaluation. Linear Discriminant Analysis and Quadratic Discriminant
6.873/HST.951 Medical Decision Support Spring 2004 Evaluation
Harvard-MIT Division of Health Sciences and Technology HST.951J: Medical Decision Support, Fall 2005 Instructors: Professor Lucila Ohno-Machado and Professor Staal Vinterbo 6.873/HST.951 Medical Decision
CptS 570 Machine Learning School of EECS Washington State University. CptS Machine Learning 1
CptS 570 Machine Learning School of EECS Washington State University CptS 570 - Machine Learning 1 IEEE Expert, October 1996 CptS 570 - Machine Learning 2 Given sample S from all possible examples D Learner
Performance Evaluation
Statistical Data Mining and Machine Learning Hilary Term 2016 Dino Sejdinovic Department of Statistics Oxford Slides and other materials available at: http://www.stats.ox.ac.uk/~sejdinov/sdmml Example:
Pointwise Exact Bootstrap Distributions of Cost Curves
Pointwise Exact Bootstrap Distributions of Cost Curves Charles Dugas and David Gadoury University of Montréal 25th ICML Helsinki July 2008 Dugas, Gadoury (U Montréal) Cost curves July 8, 2008 1 / 24 Outline
Classification. Classification is similar to regression in that the goal is to use covariates to predict on outcome.
Classification Classification is similar to regression in that the goal is to use covariates to predict on outcome. We still have a vector of covariates X. However, the response is binary (or a few classes),
Linear Discriminant Analysis Based in part on slides from textbook, slides of Susan Holmes. November 9, Statistics 202: Data Mining
Linear Discriminant Analysis Based in part on slides from textbook, slides of Susan Holmes November 9, 2012 1 / 1 Nearest centroid rule Suppose we break down our data matrix as by the labels yielding (X
CC283 Intelligent Problem Solving 28/10/2013
Machine Learning What is the research agenda? How to measure success? How to learn? Machine Learning Overview Unsupervised Learning Supervised Learning Training Testing Unseen data Data Observed x 1 x
Performance Metrics for Machine Learning. Sargur N. Srihari
Performance Metrics for Machine Learning Sargur N. srihari@cedar.buffalo.edu 1 Topics 1. Performance Metrics 2. Default Baseline Models 3. Determining whether to gather more data 4. Selecting hyperparamaters
Least Squares Classification
Least Squares Classification Stephen Boyd EE103 Stanford University November 4, 2017 Outline Classification Least squares classification Multi-class classifiers Classification 2 Classification data fitting
Dynamic Clustering-Based Estimation of Missing Values in Mixed Type Data
Dynamic Clustering-Based Estimation of Missing Values in Mixed Type Data Vadim Ayuyev, Joseph Jupin, Philip Harris and Zoran Obradovic Temple University, Philadelphia, USA 2009 Real Life Data is Often
Evaluation Metrics for Intrusion Detection Systems - A Study
Evaluation Metrics for Intrusion Detection Systems - A Study Gulshan Kumar Assistant Professor, Shaheed Bhagat Singh State Technical Campus, Ferozepur (Punjab)-India 152004 Email: gulshanahuja@gmail.com
Big Data Analytics: Evaluating Classification Performance April, 2016 R. Bohn. Some overheads from Galit Shmueli and Peter Bruce 2010
Big Data Analytics: Evaluating Classification Performance April, 2016 R. Bohn 1 Some overheads from Galit Shmueli and Peter Bruce 2010 Most accurate Best! Actual value Which is more accurate?? 2 Why Evaluate
How Many Digits In A Handshake? National Death Index Matching With Less Than Nine Digits of the Social Security Number
How Many Digits In A Handshake? National Death Index Matching With Less Than Nine Digits of the Social Security Number Bryan Sayer, Social & Scientific Systems, Inc. 8757 Georgia Ave, STE 1200, Silver
Q1 (12 points): Chap 4 Exercise 3 (a) to (f) (2 points each)
Q1 (1 points): Chap 4 Exercise 3 (a) to (f) ( points each) Given a table Table 1 Dataset for Exercise 3 Instance a 1 a a 3 Target Class 1 T T 1.0 + T T 6.0 + 3 T F 5.0-4 F F 4.0 + 5 F T 7.0-6 F T 3.0-7
Smart Home Health Analytics Information Systems University of Maryland Baltimore County
Smart Home Health Analytics Information Systems University of Maryland Baltimore County 1 IEEE Expert, October 1996 2 Given sample S from all possible examples D Learner L learns hypothesis h based on
Diagnostics. Gad Kimmel
Diagnostics Gad Kimmel Outline Introduction. Bootstrap method. Cross validation. ROC plot. Introduction Motivation Estimating properties of an estimator. Given data samples say the average. x 1, x 2,...,
BANA 7046 Data Mining I Lecture 4. Logistic Regression and Classications 1
BANA 7046 Data Mining I Lecture 4. Logistic Regression and Classications 1 Shaobo Li University of Cincinnati 1 Partially based on Hastie, et al. (2009) ESL, and James, et al. (2013) ISLR Data Mining I
CS4445 Data Mining and Knowledge Discovery in Databases. B Term 2014 Solutions Exam 2 - December 15, 2014
CS4445 Data Mining and Knowledge Discovery in Databases. B Term 2014 Solutions Exam 2 - December 15, 2014 Prof. Carolina Ruiz Department of Computer Science Worcester Polytechnic Institute NAME: Prof.
Evaluation. Andrea Passerini Machine Learning. Evaluation
Andrea Passerini passerini@disi.unitn.it Machine Learning Basic concepts requires to define performance measures to be optimized Performance of learning algorithms cannot be evaluated on entire domain
Generalized Linear Models
Generalized Linear Models Lecture 7. Models with binary response II GLM (Spring, 2018) Lecture 7 1 / 13 Existence of estimates Lemma (Claudia Czado, München, 2004) The log-likelihood ln L(β) in logistic
Linear Classifiers as Pattern Detectors
Intelligent Systems: Reasoning and Recognition James L. Crowley ENSIMAG 2 / MoSIG M1 Second Semester 2014/2015 Lesson 16 8 April 2015 Contents Linear Classifiers as Pattern Detectors Notation...2 Linear
Supplementary Information
Supplementary Information Performance measures A binary classifier, such as SVM, assigns to predicted binding sequences the positive class label (+1) and to sequences predicted as non-binding the negative
Evaluation requires to define performance measures to be optimized
Evaluation Basic concepts Evaluation requires to define performance measures to be optimized Performance of learning algorithms cannot be evaluated on entire domain (generalization error) approximation
Section 4. Test-Level Analyses
Section 4. Test-Level Analyses Test-level analyses include demographic distributions, reliability analyses, summary statistics, and decision consistency and accuracy. Demographic Distributions All eligible
Fundamentals of Similarity Search
Chapter 2 Fundamentals of Similarity Search We will now look at the fundamentals of similarity search systems, providing the background for a detailed discussion on similarity search operators in the subsequent
Data Mining and Analysis: Fundamental Concepts and Algorithms
Data Mining and Analysis: Fundamental Concepts and Algorithms dataminingbook.info Mohammed J. Zaki 1 Wagner Meira Jr. 2 1 Department of Computer Science Rensselaer Polytechnic Institute, Troy, NY, USA
Anomaly Detection. Jing Gao. SUNY Buffalo
Anomaly Detection Jing Gao SUNY Buffalo 1 Anomaly Detection Anomalies the set of objects are considerably dissimilar from the remainder of the data occur relatively infrequently when they do occur, their
Data Mining: Concepts and Techniques. (3 rd ed.) Chapter 8. Chapter 8. Classification: Basic Concepts
Data Mining: Concepts and Techniques (3 rd ed.) Chapter 8 1 Chapter 8. Classification: Basic Concepts Classification: Basic Concepts Decision Tree Induction Bayes Classification Methods Rule-Based Classification
Data Mining and Knowledge Discovery: Practice Notes
Data Mining and Knowledge Discovery: Practice Notes dr. Petra Kralj Novak Petra.Kralj.Novak@ijs.si 7.11.2017 1 Course Prof. Bojan Cestnik Data preparation Prof. Nada Lavrač: Data mining overview Advanced
Data Analytics for Social Science
Data Analytics for Social Science Johan A. Elkink School of Politics & International Relations University College Dublin 17 October 2017 Outline 1 2 3 4 5 6 Levels of measurement Discreet Continuous Nominal
Overview of IslandPick pipeline and the generation of GI datasets
Overview of IslandPick pipeline and the generation of GI datasets Predicting GIs using comparative genomics By using whole genome alignments we can identify regions that are present in one genome but not
Lecture 3 Classification, Logistic Regression
Lecture 3 Classification, Logistic Regression Fredrik Lindsten Division of Systems and Control Department of Information Technology Uppsala University. Email: fredrik.lindsten@it.uu.se F. Lindsten Summary
SUPERVISED LEARNING: INTRODUCTION TO CLASSIFICATION
SUPERVISED LEARNING: INTRODUCTION TO CLASSIFICATION 1 Outline Basic terminology Features Training and validation Model selection Error and loss measures Statistical comparison Evaluation measures 2 Terminology
Conceptual Similarity: Why, Where, How
Conceptual Similarity: Why, Where, How Michalis Sfakakis Laboratory on Digital Libraries & Electronic Publishing, Department of Archives and Library Sciences, Ionian University, Greece First Workshop on
Methods and Criteria for Model Selection. CS57300 Data Mining Fall Instructor: Bruno Ribeiro
Methods and Criteria for Model Selection CS57300 Data Mining Fall 2016 Instructor: Bruno Ribeiro Goal } Introduce classifier evaluation criteria } Introduce Bias x Variance duality } Model Assessment }
Online Advertising is Big Business
Online Advertising Online Advertising is Big Business Multiple billion dollar industry $43B in 2013 in USA, 17% increase over 2012 [PWC, Internet Advertising Bureau, April 2013] Higher revenue in USA
Part I. Linear Discriminant Analysis. Discriminant analysis. Discriminant analysis
Week 5 Based in part on slides from textbook, slides of Susan Holmes Part I Linear Discriminant Analysis October 29, 2012 1 / 1 2 / 1 Nearest centroid rule Suppose we break down our data matrix as by the
http://www.diva-portal.org This is the published version of a paper presented at NTMS'2018: The 9th IFIP International Conference on New Technologies, Mobility and Security, February 26-28, Paris, France.
Performance. Learning Classifiers. Instances x i in dataset D mapped to feature space:
Learning Classifiers Performance Instances x i in dataset D mapped to feature space: Initial Model (Assumptions) Training Process Trained Model Testing Process Tested Model Performance Performance Training
Index of Balanced Accuracy: A Performance Measure for Skewed Class Distributions
Index of Balanced Accuracy: A Performance Measure for Skewed Class Distributions V. García 1,2, R.A. Mollineda 2, and J.S. Sánchez 2 1 Lab. Reconocimiento de Patrones, Instituto Tecnológico de Toluca Av.
Social Security Administration Randomization of SSN Issuance Impact to LexisNexis Accurint XML Accurint XML Workflow and XML Tags
Social Security Administration Randomization of SSN Issuance Impact to LexisNexis Accurint XML Accurint XML Workflow and XML Tags Overview The Social Security Administration (SSA) recently announced they
Performance Measures. Sören Sonnenburg. Fraunhofer FIRST.IDA, Kekuléstr. 7, Berlin, Germany
Sören Sonnenburg Fraunhofer FIRST.IDA, Kekuléstr. 7, 2489 Berlin, Germany Roadmap: Contingency Table Scores from the Contingency Table Curves from the Contingency Table Discussion Sören Sonnenburg Contingency
Linear Classifiers as Pattern Detectors
Intelligent Systems: Reasoning and Recognition James L. Crowley ENSIMAG 2 / MoSIG M1 Second Semester 2013/2014 Lesson 18 23 April 2014 Contents Linear Classifiers as Pattern Detectors Notation...2 Linear
Hypothesis Evaluation
Hypothesis Evaluation Machine Learning Hamid Beigy Sharif University of Technology Fall 1395 Hamid Beigy (Sharif University of Technology) Hypothesis Evaluation Fall 1395 1 / 31 Table of contents 1 Introduction
The Central Limit Theorem
- The Central Limit Theorem Definition Sampling Distribution of the Mean the probability distribution of sample means, with all samples having the same sample size n. (In general, the sampling distribution
Given a feature in I 1, how to find the best match in I 2?
Feature Matching 1 Feature matching Given a feature in I 1, how to find the best match in I 2? 1. Define distance function that compares two descriptors 2. Test all the features in I 2, find the one with
SINGLE-SEQUENCE PROTEIN SECONDARY STRUCTURE PREDICTION BY NEAREST-NEIGHBOR CLASSIFICATION OF PROTEIN WORDS
SINGLE-SEQUENCE PROTEIN SECONDARY STRUCTURE PREDICTION BY NEAREST-NEIGHBOR CLASSIFICATION OF PROTEIN WORDS Item type Authors Publisher Rights text; Electronic Thesis PORFIRIO, DAVID JONATHAN The University
EXAM IN STATISTICAL MACHINE LEARNING STATISTISK MASKININLÄRNING
EXAM IN STATISTICAL MACHINE LEARNING STATISTISK MASKININLÄRNING DATE AND TIME: June 9, 2018, 09.00 14.00 RESPONSIBLE TEACHER: Andreas Svensson NUMBER OF PROBLEMS: 5 AIDING MATERIAL: Calculator, mathematical
Performance Evaluation and Hypothesis Testing
Performance Evaluation and Hypothesis Testing 1 Motivation Evaluating the performance of learning systems is important because: Learning systems are usually designed to predict the class of future unlabeled
New Prediction Methods for Tree Ensembles with Applications in Record Linkage
New Prediction Methods for Tree Ensembles with Applications in Record Linkage Samuel L. Ventura Rebecca Nugent Department of Statistics Carnegie Mellon University June 11, 2015 45th Symposium on the Interface
Data Structures in Java
Data Structures in Java Lecture 20: Algorithm Design Techniques 12/2/2015 Daniel Bauer 1 Algorithms and Problem Solving Purpose of algorithms: find solutions to problems. Data Structures provide ways of
Classifier Evaluation. Learning Curve cleval testc. The Apparent Classification Error. Error Estimation by Test Set. Classifier
Classifier Learning Curve How to estimate classifier performance. Learning curves Feature curves Rejects and ROC curves True classification error ε Bayes error ε* Sub-optimal classifier Bayes consistent
Introduction to Supervised Learning. Performance Evaluation
Introduction to Supervised Learning Performance Evaluation Marcelo S. Lauretto Escola de Artes, Ciências e Humanidades, Universidade de São Paulo marcelolauretto@usp.br Lima - Peru Performance Evaluation
Data Privacy in Biomedicine. Lecture 11b: Performance Measures for System Evaluation
Data Privacy in Biomedicine Lecture 11b: Performance Measures for System Evaluation Bradley Malin, PhD (b.malin@vanderbilt.edu) Professor of Biomedical Informatics, Biostatistics, & Computer Science Vanderbilt
Sara C. Madeira. Universidade da Beira Interior. (Thanks to Ana Teresa Freitas, IST for useful resources on this subject)
Bioinformática Sequence Alignment Pairwise Sequence Alignment Universidade da Beira Interior (Thanks to Ana Teresa Freitas, IST for useful resources on this subject) 1 16/3/29 & 23/3/29 27/4/29 Outline
Classifier performance evaluation
Classifier performance evaluation Václav Hlaváč Czech Technical University in Prague Czech Institute of Informatics, Robotics and Cybernetics 166 36 Prague 6, Jugoslávských partyzánu 1580/3, Czech Republic
Gender Classification in Speech Processing. Baiju M Nair(2011CRF3637) Geetanjali Srivastava(2012EEZ8304) Group No. 22
Gender Classification in Speech Processing Baiju M Nair(2011CRF3637) Geetanjali Srivastava(2012EEZ8304) Group No. 22 Introduction Main objective is gender classification in speech processing. Classify
Directly and Efficiently Optimizing Prediction Error and AUC of Linear Classifiers
Directly and Efficiently Optimizing Prediction Error and AUC of Linear Classifiers Hiva Ghanbari Joint work with Prof. Katya Scheinberg Industrial and Systems Engineering Department US & Mexico Workshop
Statistics for classification
AstroInformatics Statistics for classification Una rappresentazione utile è la matrice di confusione. L elemento sulla riga i e sulla colonna j è il numero assoluto oppure la percentuale di casi della
Evaluation of Intrusion Detection using a Novel Composite Metric
Evaluation of Intrusion Detection using a Novel Composite Metric T. Augustine 1, P. Vasudeva Reddy 1, P.V.G.D. Prasad Reddy 2 Department of Engineering Mathematics, A.U. College of Engineering, Andhra
Pattern Recognition and Machine Learning. Learning and Evaluation for Pattern Recognition
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2017 Lesson 1 4 October 2017 Outline Learning and Evaluation for Pattern Recognition Notation...2 1. The Pattern Recognition
ECE 661: Homework 10 Fall 2014
ECE 661: Homework 10 Fall 2014 This homework consists of the following two parts: (1) Face recognition with PCA and LDA for dimensionality reduction and the nearest-neighborhood rule for classification;
Cost-based classifier evaluation for imbalanced problems
Cost-based classifier evaluation for imbalanced problems Thomas Landgrebe, Pavel Paclík, David M.J. Tax, Serguei Verzakov, and Robert P.W. Duin Elect. Eng., Maths and Comp. Sc., Delft University of Technology,
Lecture 6: Linear Regression
Lecture 6: Linear Regression Reading: Sections 3.1-3 STATS 202: Data mining and analysis Jonathan Taylor, 10/5 Slide credits: Sergio Bacallado 1 / 30 Simple linear regression Model: y i = β 0 + β 1 x i
CHAPTER 4: PREDICTION AND ESTIMATION OF RAINFALL DURING NORTHEAST MONSOON
43 CHAPTER 4: PREDICTION AND ESTIMATION OF RAINFALL DURING NORTHEAST MONSOON After analyzing the weather patterns and forecast of seasonal rainfall for Cauvery delta region, a method to predict the dry
Model generation and model selection in credit scoring
Model generation and model selection in credit scoring Vadim STRIJOV Russian Academy of Sciences Computing Center EURO 2010 Lisbon July 14 th The workflow Client s application & history Client s score:
An Overview of Outlier Detection Techniques and Applications
Machine Learning Rhein-Neckar Meetup An Overview of Outlier Detection Techniques and Applications Ying Gu connygy@gmail.com 28.02.2016 Anomaly/Outlier Detection What are anomalies/outliers? The set of
Reducing False Alarm Rate in Anomaly Detection with Layered Filtering
Reducing False Alarm Rate in Anomaly Detection with Layered Filtering Rafa l Pokrywka 1,2 1 Institute of Computer Science AGH University of Science and Technology al. Mickiewicza 30, 30-059 Kraków, Poland
Ensemble Methods. NLP ML Web! Fall 2013! Andrew Rosenberg! TA/Grader: David Guy Brizan
Ensemble Methods NLP ML Web! Fall 2013! Andrew Rosenberg! TA/Grader: David Guy Brizan How do you make a decision? What do you want for lunch today?! What did you have last night?! What are your favorite
Efficient High-Similarity String Comparison: The Waterfall Algorithm
Efficient High-Similarity String Comparison: The Waterfall Algorithm Alexander Tiskin Department of Computer Science University of Warwick http://go.warwick.ac.uk/alextiskin Alexander Tiskin (Warwick)
Introduction to Logistic Regression
Introduction to Logistic Regression Problem & Data Overview Primary Research Questions: 1. What are the risk factors associated with CHD? Regression Questions: 1. What is Y? 2. What is X? Did player develop
Online Passive-Aggressive Algorithms. Tirgul 11
Online Passive-Aggressive Algorithms Tirgul 11 Multi-Label Classification 2 Multilabel Problem: Example Mapping Apps to smart folders: Assign an installed app to one or more folders Candy Crush Saga 3
Efficient Approximate Entity Matching Using Jaro-Winkler Distance
Efficient Approximate Entity Matching Using Jaro-Winkler Distance Yaoshu Wang (B), Jianbin Qin, and Wei Wang School of Computer Science and Engineering, Univeristy of New South Wales, Sydney, Australia
Probabilistic Spelling Correction CE-324: Modern Information Retrieval Sharif University of Technology
Probabilistic Spelling Correction CE-324: Modern Information Retrieval Sharif University of Technology M. Soleymani Fall 2016 Most slides have been adapted from: Profs. Manning, Nayak & Raghavan lectures
MACHINE LEARNING ADVANCED MACHINE LEARNING
MACHINE LEARNING ADVANCED MACHINE LEARNING Recap of Important Notions on Estimation of Probability Density Functions 2 2 MACHINE LEARNING Overview Definition pdf Definition joint, condition, marginal,
Analysis and Design of Algorithms Dynamic Programming
Analysis and Design of Algorithms Dynamic Programming Lecture Notes by Dr. Wang, Rui Fall 2008 Department of Computer Science Ocean University of China November 6, 2009 Introduction 2 Introduction..................................................................
Multiple regression: Categorical dependent variables
Multiple : Categorical Johan A. Elkink School of Politics & International Relations University College Dublin 28 November 2016 1 2 3 4 Outline 1 2 3 4 models models have a variable consisting of two categories.
Euclid Contest Tuesday, April 12, 2011
The ENTRE for EDUTION in MTHEMTIS and OMPUTING wwwcemcuwaterlooca Euclid ontest Tuesday, pril 12, 2011 Time: 2 1 2 hours c 2011 entre for Education in Mathematics and omputing alculators are permitted,
Fast Comparison of Software Birthmarks for Detecting the Theft with the Search Engine
2016 4th Intl Conf on Applied Computing and Information Technology/3rd Intl Conf on Computational Science/Intelligence and Applied Informatics/1st Intl Conf on Big Data, Cloud Computing, Data Science &
Implementing Approximate Regularities
Implementing Approximate Regularities Manolis Christodoulakis Costas S. Iliopoulos Department of Computer Science King s College London Kunsoo Park School of Computer Science and Engineering, Seoul National
Hypothesis tests
6.1 6.4 Hypothesis tests Prof. Tesler Math 186 February 26, 2014 Prof. Tesler 6.1 6.4 Hypothesis tests Math 186 / February 26, 2014 1 / 41 6.1 6.2 Intro to hypothesis tests and decision rules Hypothesis
Lecture 4: September 19
CSCI1810: Computational Molecular Biology Fall 2017 Lecture 4: September 19 Lecturer: Sorin Istrail Scribe: Cyrus Cousins Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes
2013 Eric Pitman Summer Workshop in Computational Science....an introduction to R, statistics, programming, and getting to know datasets
2013 Eric Pitman Summer Workshop in Computational Science...an introduction to R, statistics, programming, and getting to know datasets Introducing the Workshop Project Here's what we'll cover: The story