Optical Character Recognition of Jutakshars within Devanagari Script
|
|
|
- Shawn Lawrence
- 5 years ago
- Views:
Transcription
1 Optical Character Recognition of Jutakshars within Devanagari Script Sheallika Singh Shreesh Ladha Supervised by : Dr. Harish Karnick, Dr. Amit Mitra UGP Presentation, 10 April 2016 OCR of Jutakshars within Devanagari Script UGP 1 / 35
2 Basic working of OCR Convert printed/scanned text into editable text OCR of Jutakshars within Devanagari Script UGP 2 / 35
3 Basic working of OCR Convert printed/scanned text into editable text Starts off with some basic preprocessing of the image for uniformity OCR of Jutakshars within Devanagari Script UGP 2 / 35
4 Basic working of OCR Convert printed/scanned text into editable text Starts off with some basic preprocessing of the image for uniformity Segmentation of lines, words, characters based on their horizontal and vertical histograms of pixel intensities OCR of Jutakshars within Devanagari Script UGP 2 / 35
5 Basic working of OCR Convert printed/scanned text into editable text Starts off with some basic preprocessing of the image for uniformity Segmentation of lines, words, characters based on their horizontal and vertical histograms of pixel intensities Classifiers are then used for prediction OCR of Jutakshars within Devanagari Script UGP 2 / 35
6 Basic working of OCR Convert printed/scanned text into editable text Starts off with some basic preprocessing of the image for uniformity Segmentation of lines, words, characters based on their horizontal and vertical histograms of pixel intensities Classifiers are then used for prediction Composition using language rules and n-gram models OCR of Jutakshars within Devanagari Script UGP 2 / 35
7 Figure: Horizontal and vertical histograms OCR of Jutakshars within Devanagari Script UGP 3 / 35
8 Figure: Example of Segmentatin OCR of Jutakshars within Devanagari Script UGP 4 / 35
9 Challenges with Devanagari Robust OCR systems already in place for Roman Scripts OCR of Jutakshars within Devanagari Script UGP 5 / 35
10 Challenges with Devanagari Robust OCR systems already in place for Roman Scripts Lack of such software for Hindi Languages because of highly complicated structure and composition OCR of Jutakshars within Devanagari Script UGP 5 / 35
11 Challenges with Devanagari Robust OCR systems already in place for Roman Scripts Lack of such software for Hindi Languages because of highly complicated structure and composition An eg. of such composition is a form of character conjunction referred to as Jutakshar OCR of Jutakshars within Devanagari Script UGP 5 / 35
12 Figure: Classes of Jutakshars. *Note the model was developed to understand Jutakshars similar in type with the first four OCR of Jutakshars within Devanagari Script UGP 6 / 35
13 Our Work Words containing Jutakshar are quite popular and can be found in many document OCR of Jutakshars within Devanagari Script UGP 7 / 35
14 Our Work Words containing Jutakshar are quite popular and can be found in many document Most OCR systems do not take care to handle such characters, which in turn reduces their precision OCR of Jutakshars within Devanagari Script UGP 7 / 35
15 Our Work Words containing Jutakshar are quite popular and can be found in many document Most OCR systems do not take care to handle such characters, which in turn reduces their precision Low accuracy due to Jutakshars being recognized as a single character OCR of Jutakshars within Devanagari Script UGP 7 / 35
16 Our Work Words containing Jutakshar are quite popular and can be found in many document Most OCR systems do not take care to handle such characters, which in turn reduces their precision Low accuracy due to Jutakshars being recognized as a single character We have tried to build upon existing frameworks to better handle these special strings OCR of Jutakshars within Devanagari Script UGP 7 / 35
17 Figure: Where segmentation fails OCR of Jutakshars within Devanagari Script UGP 8 / 35
18 Approach Jutakshar Detection : Figure out whether a character is a jutakshar or not OCR of Jutakshars within Devanagari Script UGP 9 / 35
19 Approach Jutakshar Detection : Figure out whether a character is a jutakshar or not Jutakshar Identification : Figure out the jutakshar and in turn the entire word OCR of Jutakshars within Devanagari Script UGP 9 / 35
20 Jutakshar Detection Created dataset of jutakshars using 8 fonts and added gaussian noise OCR of Jutakshars within Devanagari Script UGP 10 / 35
21 Jutakshar Detection Created dataset of jutakshars using 8 fonts and added gaussian noise Trained classifiers to predict whether a given character was a Jutakshar or not OCR of Jutakshars within Devanagari Script UGP 10 / 35
22 Jutakshar Detection Created dataset of jutakshars using 8 fonts and added gaussian noise Trained classifiers to predict whether a given character was a Jutakshar or not Experimented with three different types of models OCR of Jutakshars within Devanagari Script UGP 10 / 35
23 Jutakshar Detection Model I : SVM trained using hog features OCR of Jutakshars within Devanagari Script UGP 11 / 35
24 Jutakshar Detection Model I : SVM trained using hog features Model II : Logistic regression using features extracted from penultimate layer of a pretrained convolutional neural network OCR of Jutakshars within Devanagari Script UGP 11 / 35
25 Jutakshar Detection Model I : SVM trained using hog features Model II : Logistic regression using features extracted from penultimate layer of a pretrained convolutional neural network Model III : Classification using the width of the character boxes OCR of Jutakshars within Devanagari Script UGP 11 / 35
26 Figure: Comparison of howocr different of Jutakshars models within Devanagari work for Script Jutakshar Detection. UGP Model 12 / 35
27 Jutakshar Identification Using the previous model, the character boxes predicted as containing jutakshars, are now split into the corresponding half and full character OCR of Jutakshars within Devanagari Script UGP 13 / 35
28 Jutakshar Identification Using the previous model, the character boxes predicted as containing jutakshars, are now split into the corresponding half and full character Used vertical histograms for splitting but restricted our search in the middle part of the character OCR of Jutakshars within Devanagari Script UGP 13 / 35
29 Jutakshar Identification Using the previous model, the character boxes predicted as containing jutakshars, are now split into the corresponding half and full character Used vertical histograms for splitting but restricted our search in the middle part of the character The position of the minima was used to split the the Jutakshar into the half and the full character forming it OCR of Jutakshars within Devanagari Script UGP 13 / 35
30 Figure: Example showing how splitting works on Jutakshars OCR of Jutakshars within Devanagari Script UGP 14 / 35
31 Jutakshar Identification Segmented word is passed onto the character classifiers to predict the respective characters OCR of Jutakshars within Devanagari Script UGP 15 / 35
32 Jutakshar Identification Segmented word is passed onto the character classifiers to predict the respective characters The predicted word is passed to a dictionary which gives a list of suggested correct words for an incorrect word OCR of Jutakshars within Devanagari Script UGP 15 / 35
33 Jutakshar Identification Segmented word is passed onto the character classifiers to predict the respective characters The predicted word is passed to a dictionary which gives a list of suggested correct words for an incorrect word A heuristic is defined to find the correct prediction from the suggested list OCR of Jutakshars within Devanagari Script UGP 15 / 35
34 Jutakshar Identification Suggested list of words refined to have only words which contain Jutskshars and having length as close as possible OCR of Jutakshars within Devanagari Script UGP 16 / 35
35 Jutakshar Identification Suggested list of words refined to have only words which contain Jutskshars and having length as close as possible Levenshtein distance (computes the minimum number of steps required for converting one string into another by insertions, deletions or substitutions of single characters) between the predicted and the suggested word OCR of Jutakshars within Devanagari Script UGP 16 / 35
36 Jutakshar Identification Suggested list of words refined to have only words which contain Jutskshars and having length as close as possible Levenshtein distance (computes the minimum number of steps required for converting one string into another by insertions, deletions or substitutions of single characters) between the predicted and the suggested word Number of substrings that match in the predicted word with the jutakshars removed and the suggested word OCR of Jutakshars within Devanagari Script UGP 16 / 35
37 Jutakshar Identification Suggested list of words refined to have only words which contain Jutskshars and having length as close as possible Levenshtein distance (computes the minimum number of steps required for converting one string into another by insertions, deletions or substitutions of single characters) between the predicted and the suggested word Number of substrings that match in the predicted word with the jutakshars removed and the suggested word Length of the longest common substring between the predicted word with the jutakshars removed and the suggested word in the list OCR of Jutakshars within Devanagari Script UGP 16 / 35
38 Figure: Example of suggestions returned by the dictionary OCR of Jutakshars within Devanagari Script UGP 17 / 35
39 Figure: Correction in the predicted word after applying our heuristic OCR of Jutakshars within Devanagari Script UGP 18 / 35
40 Results for Jutakshar Detection Model Figure: Plot of accuracies of Jutakshar detection model on different fonts and documents OCR of Jutakshars within Devanagari Script UGP 19 / 35
41 Results : Jutakshar Detection Document Total # Words # True # False Words containing Positives Positives Jutakshars Doc Doc Doc Doc Table: Jutakshar Detection (Font 1) OCR of Jutakshars within Devanagari Script UGP 20 / 35
42 Results : Jutakshar Detection Document Total # Words # True # False Words containing Positives Positives Jutakshars Doc Doc Doc Doc Table: Jutakshar Detection (Font 2) OCR of Jutakshars within Devanagari Script UGP 21 / 35
43 Results : Jutakshar Detection Document Total # Words # True # False Words containing Positives Positives Jutakshars Doc Doc Doc Doc Table: Jutakshar Detection (Font 3) OCR of Jutakshars within Devanagari Script UGP 22 / 35
44 Results : Jutakshar Detection Document Total # Words # True # False Words containing Positives Positives Jutakshars Doc Doc Doc Doc Table: Jutakshar Detection (Font 4) OCR of Jutakshars within Devanagari Script UGP 23 / 35
45 Results : Jutakshar Detection Document Total # Words # True # False Words containing Positives Positives Jutakshars Doc Doc Doc Doc Table: Jutakshar Detection (Font 5) OCR of Jutakshars within Devanagari Script UGP 24 / 35
46 Results : Jutakshar Detection False Positive type of cases OCR of Jutakshars within Devanagari Script UGP 25 / 35
47 OCR Evaluation OCR of Jutakshars within Devanagari Script UGP 26 / 35
48 Error rates WER: #Erroneous words divided by total word count in ground-truth text CER: #Erroneous characters divided by total character count in ground-truth text Evaluation tool: OCR of Jutakshars within Devanagari Script UGP 27 / 35
49 Error rates WER: #Erroneous words divided by total word count in ground-truth text CER: #Erroneous characters divided by total character count in ground-truth text Evaluation tool: Mathematically, WER(%) = (S w + I w + D w ) 100 N w CER(%) = (S c + I c + D c ) 100 N c OCR of Jutakshars within Devanagari Script UGP 27 / 35
50 Results : Jutakshar Identification Plot for Character Error rates with which word containing jutakshars are identified OCR of Jutakshars within Devanagari Script UGP 28 / 35
51 Overall Accuracies Document CER(%) WER(%) CER(%) WER(%) (Prev Model) (Prev Model) (Our Model) (Our Model) Doc Doc Doc Doc Overall Table: OCR Accuracy : Comparing Models (Font 1) OCR of Jutakshars within Devanagari Script UGP 29 / 35
52 Overall Accuracies Document CER(%) WER(%) CER(%) WER(%) (Prev Model) (Prev Model) (Our Model) (Our Model) Doc Doc Doc Doc Overall Table: OCR Accuracy : Comparing Models (Font 2) OCR of Jutakshars within Devanagari Script UGP 30 / 35
53 Overall Accuracies Document CER(%) WER(%) CER(%) WER(%) (Prev Model) (Prev Model) (Our Model) (Our Model) Doc Doc Doc Doc Overall Table: OCR Accuracy : Comparing Models (Font 3) OCR of Jutakshars within Devanagari Script UGP 31 / 35
54 Overall Accuracies Document CER(%) WER(%) CER(%) WER(%) (Prev Model) (Prev Model) (Our Model) (Our Model) Doc Doc Doc Doc Overall Table: OCR Accuracy : Comparing Models (Font 4) OCR of Jutakshars within Devanagari Script UGP 32 / 35
55 Overall Accuracies Document CER(%) WER(%) CER(%) WER(%) (Prev Model) (Prev Model) (Our Model) (Our Model) Doc Doc Doc Doc Overall Table: OCR Accuracy : Comparing Models (Font 5) OCR of Jutakshars within Devanagari Script UGP 33 / 35
56 Future Work Our model relies heavily on the dictionary. Hence a more powerful dictionary would reduce quite a few errors OCR of Jutakshars within Devanagari Script UGP 34 / 35
57 Future Work Our model relies heavily on the dictionary. Hence a more powerful dictionary would reduce quite a few errors The dataset can be made more representative by adding more fonts and possible jutakshars OCR of Jutakshars within Devanagari Script UGP 34 / 35
58 Future Work Our model relies heavily on the dictionary. Hence a more powerful dictionary would reduce quite a few errors The dataset can be made more representative by adding more fonts and possible jutakshars Two deepnet models could be trained - One for all half characters and the other for all different types of full characters OCR of Jutakshars within Devanagari Script UGP 34 / 35
59 Questions? OCR of Jutakshars within Devanagari Script UGP 35 / 35
Feature Extraction Using Zernike Moments
Feature Extraction Using Zernike Moments P. Bhaskara Rao Department of Electronics and Communication Engineering ST.Peter's Engineeing college,hyderabad,andhra Pradesh,India D.Vara Prasad Department of
Learning theory. Ensemble methods. Boosting. Boosting: history
Learning theory Probability distribution P over X {0, 1}; let (X, Y ) P. We get S := {(x i, y i )} n i=1, an iid sample from P. Ensemble methods Goal: Fix ɛ, δ (0, 1). With probability at least 1 δ (over
Neural Networks & Fuzzy Logic. Alphabet Identification: A MATLAB Implementation using a feed-forward neural network.
Neural Networks & Fuzzy Logic Alphabet Identification: A MATLAB Implementation using a feed-forward neural network Sudharshan Suresh Problem Definition: To identify an input alphabet between A-Z using
When Dictionary Learning Meets Classification
When Dictionary Learning Meets Classification Bufford, Teresa 1 Chen, Yuxin 2 Horning, Mitchell 3 Shee, Liberty 1 Mentor: Professor Yohann Tendero 1 UCLA 2 Dalhousie University 3 Harvey Mudd College August
Neural networks and optimization
Neural networks and optimization Nicolas Le Roux INRIA 8 Nov 2011 Nicolas Le Roux (INRIA) Neural networks and optimization 8 Nov 2011 1 / 80 1 Introduction 2 Linear classifier 3 Convolutional neural networks
Anticipating Visual Representations from Unlabeled Data. Carl Vondrick, Hamed Pirsiavash, Antonio Torralba
Anticipating Visual Representations from Unlabeled Data Carl Vondrick, Hamed Pirsiavash, Antonio Torralba Overview Problem Key Insight Methods Experiments Problem: Predict future actions and objects Image
COMS 4771 Lecture Boosting 1 / 16
COMS 4771 Lecture 12 1. Boosting 1 / 16 Boosting What is boosting? Boosting: Using a learning algorithm that provides rough rules-of-thumb to construct a very accurate predictor. 3 / 16 What is boosting?
Neural networks and optimization
Neural networks and optimization Nicolas Le Roux Criteo 18/05/15 Nicolas Le Roux (Criteo) Neural networks and optimization 18/05/15 1 / 85 1 Introduction 2 Deep networks 3 Optimization 4 Convolutional
Fast Logistic Regression for Text Categorization with Variable-Length N-grams
Fast Logistic Regression for Text Categorization with Variable-Length N-grams Georgiana Ifrim *, Gökhan Bakır +, Gerhard Weikum * * Max-Planck Institute for Informatics Saarbrücken, Germany + Google Switzerland
Using a Hopfield Network: A Nuts and Bolts Approach
Using a Hopfield Network: A Nuts and Bolts Approach November 4, 2013 Gershon Wolfe, Ph.D. Hopfield Model as Applied to Classification Hopfield network Training the network Updating nodes Sequencing of
10/05/2016. Computational Methods for Data Analysis. Massimo Poesio SUPPORT VECTOR MACHINES. Support Vector Machines Linear classifiers
Computational Methods for Data Analysis Massimo Poesio SUPPORT VECTOR MACHINES Support Vector Machines Linear classifiers 1 Linear Classifiers denotes +1 denotes -1 w x + b>0 f(x,w,b) = sign(w x + b) How
Article from. Predictive Analytics and Futurism. July 2016 Issue 13
Article from Predictive Analytics and Futurism July 2016 Issue 13 Regression and Classification: A Deeper Look By Jeff Heaton Classification and regression are the two most common forms of models fitted
Support Vector Machines (SVM) in bioinformatics. Day 1: Introduction to SVM
1 Support Vector Machines (SVM) in bioinformatics Day 1: Introduction to SVM Jean-Philippe Vert Bioinformatics Center, Kyoto University, Japan Jean-Philippe.Vert@mines.org Human Genome Center, University
Machine Learning 2010
Machine Learning 2010 Michael M Richter Support Vector Machines Email: mrichter@ucalgary.ca 1 - Topic This chapter deals with concept learning the numerical way. That means all concepts, problems and decisions
Deep Learning for Natural Language Processing. Sidharth Mudgal April 4, 2017
Deep Learning for Natural Language Processing Sidharth Mudgal April 4, 2017 Table of contents 1. Intro 2. Word Vectors 3. Word2Vec 4. Char Level Word Embeddings 5. Application: Entity Matching 6. Conclusion
Machine learning comes from Bayesian decision theory in statistics. There we want to minimize the expected value of the loss function.
Bayesian learning: Machine learning comes from Bayesian decision theory in statistics. There we want to minimize the expected value of the loss function. Let y be the true label and y be the predicted
EECS490: Digital Image Processing. Lecture #26
Lecture #26 Moments; invariant moments Eigenvector, principal component analysis Boundary coding Image primitives Image representation: trees, graphs Object recognition and classes Minimum distance classifiers
Machine Learning Basics
Security and Fairness of Deep Learning Machine Learning Basics Anupam Datta CMU Spring 2019 Image Classification Image Classification Image classification pipeline Input: A training set of N images, each
Deviations from linear separability. Kernel methods. Basis expansion for quadratic boundaries. Adding new features Systematic deviation
Deviations from linear separability Kernel methods CSE 250B Noise Find a separator that minimizes a convex loss function related to the number of mistakes. e.g. SVM, logistic regression. Systematic deviation
Kernel methods CSE 250B
Kernel methods CSE 250B Deviations from linear separability Noise Find a separator that minimizes a convex loss function related to the number of mistakes. e.g. SVM, logistic regression. Deviations from
FARSI CHARACTER RECOGNITION USING NEW HYBRID FEATURE EXTRACTION METHODS
FARSI CHARACTER RECOGNITION USING NEW HYBRID FEATURE EXTRACTION METHODS Fataneh Alavipour 1 and Ali Broumandnia 2 1 Department of Electrical, Computer and Biomedical Engineering, Qazvin branch, Islamic
A Novel Rejection Measurement in Handwritten Numeral Recognition Based on Linear Discriminant Analysis
009 0th International Conference on Document Analysis and Recognition A Novel Rejection easurement in Handwritten Numeral Recognition Based on Linear Discriminant Analysis Chun Lei He Louisa Lam Ching
Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers
Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers Erin Allwein, Robert Schapire and Yoram Singer Journal of Machine Learning Research, 1:113-141, 000 CSE 54: Seminar on Learning
The Perceptron. Volker Tresp Summer 2014
The Perceptron Volker Tresp Summer 2014 1 Introduction One of the first serious learning machines Most important elements in learning tasks Collection and preprocessing of training data Definition of a
Mathematical Tools for Neuroscience (NEU 314) Princeton University, Spring 2016 Jonathan Pillow. Homework 8: Logistic Regression & Information Theory
Mathematical Tools for Neuroscience (NEU 34) Princeton University, Spring 206 Jonathan Pillow Homework 8: Logistic Regression & Information Theory Due: Tuesday, April 26, 9:59am Optimization Toolbox One
Holdout and Cross-Validation Methods Overfitting Avoidance
Holdout and Cross-Validation Methods Overfitting Avoidance Decision Trees Reduce error pruning Cost-complexity pruning Neural Networks Early stopping Adjusting Regularizers via Cross-Validation Nearest
CS:4330 Theory of Computation Spring Regular Languages. Finite Automata and Regular Expressions. Haniel Barbosa
CS:4330 Theory of Computation Spring 2018 Regular Languages Finite Automata and Regular Expressions Haniel Barbosa Readings for this lecture Chapter 1 of [Sipser 1996], 3rd edition. Sections 1.1 and 1.3.
CS4495/6495 Introduction to Computer Vision. 8C-L3 Support Vector Machines
CS4495/6495 Introduction to Computer Vision 8C-L3 Support Vector Machines Discriminative classifiers Discriminative classifiers find a division (surface) in feature space that separates the classes Several
CS 188: Artificial Intelligence Spring Announcements
CS 188: Artificial Intelligence Spring 2010 Lecture 24: Perceptrons and More! 4/22/2010 Pieter Abbeel UC Berkeley Slides adapted from Dan Klein Announcements W7 due tonight [this is your last written for
Reliability of Seismic Data for Hydrocarbon Reservoir Characterization
Reliability of Seismic Data for Hydrocarbon Reservoir Characterization Geetartha Dutta (gdutta@stanford.edu) December 10, 2015 Abstract Seismic data helps in better characterization of hydrocarbon reservoirs.
Neural Networks biological neuron artificial neuron 1
Neural Networks biological neuron artificial neuron 1 A two-layer neural network Output layer (activation represents classification) Weighted connections Hidden layer ( internal representation ) Input
Introduction to Machine Learning
Introduction to Machine Learning CS4731 Dr. Mihail Fall 2017 Slide content based on books by Bishop and Barber. https://www.microsoft.com/en-us/research/people/cmbishop/ http://web4.cs.ucl.ac.uk/staff/d.barber/pmwiki/pmwiki.php?n=brml.homepage
Sparse Proteomics Analysis (SPA)
Sparse Proteomics Analysis (SPA) Toward a Mathematical Theory for Feature Selection from Forward Models Martin Genzel Technische Universität Berlin Winter School on Compressed Sensing December 5, 2015
Kaggle.
Administrivia Mini-project 2 due April 7, in class implement multi-class reductions, naive bayes, kernel perceptron, multi-class logistic regression and two layer neural networks training set: Project
Metric Learning. 16 th Feb 2017 Rahul Dey Anurag Chowdhury
Metric Learning 16 th Feb 2017 Rahul Dey Anurag Chowdhury 1 Presentation based on Bellet, Aurélien, Amaury Habrard, and Marc Sebban. "A survey on metric learning for feature vectors and structured data."
Support Vector Machines. CAP 5610: Machine Learning Instructor: Guo-Jun QI
Support Vector Machines CAP 5610: Machine Learning Instructor: Guo-Jun QI 1 Linear Classifier Naive Bayes Assume each attribute is drawn from Gaussian distribution with the same variance Generative model:
Noisy Subsequence Recognition Using Constrained String Editing Involving Substitutions, Insertions, Deletions and Generalized Transpositions 1
Noisy Subsequence Recognition Using Constrained String Editing Involving Substitutions, Insertions, Deletions and Generalized Transpositions 1 B. J. Oommen and R. K. S. Loke School of Computer Science
Off-line Handwritten Kannada Text Recognition using Support Vector Machine using Zernike Moments
128 IJCSNS International Journal of Computer Science and Network Security, VOL.11 No.7, July 2011 Off-line Handwritten Kannada Text Recognition using Support Vector Machine using Zernike Moments Thungamani.M
Linear, threshold units. Linear Discriminant Functions and Support Vector Machines. Biometrics CSE 190 Lecture 11. X i : inputs W i : weights
Linear Discriminant Functions and Support Vector Machines Linear, threshold units CSE19, Winter 11 Biometrics CSE 19 Lecture 11 1 X i : inputs W i : weights θ : threshold 3 4 5 1 6 7 Courtesy of University
The Perceptron. Volker Tresp Summer 2016
The Perceptron Volker Tresp Summer 2016 1 Elements in Learning Tasks Collection, cleaning and preprocessing of training data Definition of a class of learning models. Often defined by the free model parameters
Mitosis Detection in Breast Cancer Histology Images with Multi Column Deep Neural Networks
Mitosis Detection in Breast Cancer Histology Images with Multi Column Deep Neural Networks IDSIA, Lugano, Switzerland dan.ciresan@gmail.com Dan C. Cireşan and Alessandro Giusti DNN for Visual Pattern Recognition
Pattern Recognition and Machine Learning. Perceptrons and Support Vector machines
Pattern Recognition and Machine Learning James L. Crowley ENSIMAG 3 - MMIS Fall Semester 2016 Lessons 6 10 Jan 2017 Outline Perceptrons and Support Vector machines Notation... 2 Perceptrons... 3 History...3
Linear Classification and SVM. Dr. Xin Zhang
Linear Classification and SVM Dr. Xin Zhang Email: eexinzhang@scut.edu.cn What is linear classification? Classification is intrinsically non-linear It puts non-identical things in the same class, so a
Introduction to Machine Learning Midterm Exam
10-701 Introduction to Machine Learning Midterm Exam Instructors: Eric Xing, Ziv Bar-Joseph 17 November, 2015 There are 11 questions, for a total of 100 points. This exam is open book, open notes, but
Homework 1 Solutions Probability, Maximum Likelihood Estimation (MLE), Bayes Rule, knn
Homework 1 Solutions Probability, Maximum Likelihood Estimation (MLE), Bayes Rule, knn CMU 10-701: Machine Learning (Fall 2016) https://piazza.com/class/is95mzbrvpn63d OUT: September 13th DUE: September
Introduction to Neural Networks
CUONG TUAN NGUYEN SEIJI HOTTA MASAKI NAKAGAWA Tokyo University of Agriculture and Technology Copyright by Nguyen, Hotta and Nakagawa 1 Pattern classification Which category of an input? Example: Character
Integrating Knowledge Sources in Devanagari Text Recognition System
500 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART A: SYSTEMS AND HUMANS, VOL. 30, NO. 4, JULY 2000 [16] U. Ophthalmology Branch, Clinical sciences division, Aerospace Ophtalmology, 1988, vol.
ECE 521. Lecture 11 (not on midterm material) 13 February K-means clustering, Dimensionality reduction
ECE 521 Lecture 11 (not on midterm material) 13 February 2017 K-means clustering, Dimensionality reduction With thanks to Ruslan Salakhutdinov for an earlier version of the slides Overview K-means clustering
Pattern Recognition Problem. Pattern Recognition Problems. Pattern Recognition Problems. Pattern Recognition: OCR. Pattern Recognition Books
Introduction to Statistical Pattern Recognition Pattern Recognition Problem R.P.W. Duin Pattern Recognition Group Delft University of Technology The Netherlands prcourse@prtools.org What is this? What
Lecture 4 Discriminant Analysis, k-nearest Neighbors
Lecture 4 Discriminant Analysis, k-nearest Neighbors Fredrik Lindsten Division of Systems and Control Department of Information Technology Uppsala University. Email: fredrik.lindsten@it.uu.se fredrik.lindsten@it.uu.se
Sparse Kernel Machines - SVM
Sparse Kernel Machines - SVM Henrik I. Christensen Robotics & Intelligent Machines @ GT Georgia Institute of Technology, Atlanta, GA 30332-0280 hic@cc.gatech.edu Henrik I. Christensen (RIM@GT) Support
Machine Learning And Applications: Supervised Learning-SVM
Machine Learning And Applications: Supervised Learning-SVM Raphaël Bournhonesque École Normale Supérieure de Lyon, Lyon, France raphael.bournhonesque@ens-lyon.fr 1 Supervised vs unsupervised learning Machine
Bounding the Probability of Error for High Precision Recognition
Bounding the Probability of Error for High Precision Recognition Andrew Kae Department of Computer Science University of Massachusetts Amherst Amherst, MA 01003 akae@cs.umass.edu Gary B. Huang Department
Machine Learning Methods for Radio Host Cross-Identification with Crowdsourced Labels
Machine Learning Methods for Radio Host Cross-Identification with Crowdsourced Labels Matthew Alger (ANU), Julie Banfield (ANU/WSU), Cheng Soon Ong (Data61/ANU), Ivy Wong (ICRAR/UWA) Slides: http://www.mso.anu.edu.au/~alger/sparcs-vii
Neural Networks. David Rosenberg. July 26, New York University. David Rosenberg (New York University) DS-GA 1003 July 26, / 35
Neural Networks David Rosenberg New York University July 26, 2017 David Rosenberg (New York University) DS-GA 1003 July 26, 2017 1 / 35 Neural Networks Overview Objectives What are neural networks? How
Perceptron Revisited: Linear Separators. Support Vector Machines
Support Vector Machines Perceptron Revisited: Linear Separators Binary classification can be viewed as the task of separating classes in feature space: w T x + b > 0 w T x + b = 0 w T x + b < 0 Department
Unsupervised Anomaly Detection for High Dimensional Data
Unsupervised Anomaly Detection for High Dimensional Data Department of Mathematics, Rowan University. July 19th, 2013 International Workshop in Sequential Methodologies (IWSM-2013) Outline of Talk Motivation
Application of Jaro-Winkler String Comparator in Enhancing Veterans Administrative Records
Application of Jaro-Winkler String Comparator in Enhancing Veterans Administrative Records Hyo Park, Eddie Thomas, Pheak Lim The Office of Data Governance and Analytics Department of Veterans Affairs FCSM,
Machine Learning Overview
Machine Learning Overview Sargur N. Srihari University at Buffalo, State University of New York USA 1 Outline 1. What is Machine Learning (ML)? 2. Types of Information Processing Problems Solved 1. Regression
Automatic detection and of dipoles in large area SQUID magnetometry
Automatic detection and of dipoles in large area SQUID magnetometry Lisa Qian December 4, Introduction. Scanning SQUID magnetometry Scanning SQUID magnetometry is a powerful tool for metrology of individual
Statistical Filters for Crowd Image Analysis
Statistical Filters for Crowd Image Analysis Ákos Utasi, Ákos Kiss and Tamás Szirányi Distributed Events Analysis Research Group, Computer and Automation Research Institute H-1111 Budapest, Kende utca
Handwritten English Character Recognition using Pixel Density Gradient Method
International Journal of Computer Science and Engineering Open Access Research Paper Volume-2, Issue-3 E-ISSN: 2347-2693 Handwritten English Character Recognition using Pixel Density Gradient Method Rakesh
Contents Lecture 4. Lecture 4 Linear Discriminant Analysis. Summary of Lecture 3 (II/II) Summary of Lecture 3 (I/II)
Contents Lecture Lecture Linear Discriminant Analysis Fredrik Lindsten Division of Systems and Control Department of Information Technology Uppsala University Email: fredriklindsten@ituuse Summary of lecture
Support Vector Machine I
Support Vector Machine I Jia-Bin Huang ECE-5424G / CS-5824 Virginia Tech Spring 2019 Administrative Please use piazza. No emails. HW 0 grades are back. Re-grade request for one week. HW 1 due soon. HW
Learning with kernels and SVM
Learning with kernels and SVM Šámalova chata, 23. května, 2006 Petra Kudová Outline Introduction Binary classification Learning with Kernels Support Vector Machines Demo Conclusion Learning from data find
Last Time. Today. Bayesian Learning. The Distributions We Love. CSE 446 Gaussian Naïve Bayes & Logistic Regression
CSE 446 Gaussian Naïve Bayes & Logistic Regression Winter 22 Dan Weld Learning Gaussians Naïve Bayes Last Time Gaussians Naïve Bayes Logistic Regression Today Some slides from Carlos Guestrin, Luke Zettlemoyer
Scientific Data Mining: Why is it Difficult? Sapphire: using data mining techniques to address the data overload problem
Scientific Data Mining: Why is it Difficult? Chandrika Kamath June 25, 2008 MMDS 2008: Workshop on Algorithms for Modern Massive Data Sets LLNL-PRES-404920: This work performed under the auspices of the
Linear vs Non-linear classifier. CS789: Machine Learning and Neural Network. Introduction
Linear vs Non-linear classifier CS789: Machine Learning and Neural Network Support Vector Machine Jakramate Bootkrajang Department of Computer Science Chiang Mai University Linear classifier is in the
Naïve Bayes. Vibhav Gogate The University of Texas at Dallas
Naïve Bayes Vibhav Gogate The University of Texas at Dallas Supervised Learning of Classifiers Find f Given: Training set {(x i, y i ) i = 1 n} Find: A good approximation to f : X Y Examples: what are
PAC-learning, VC Dimension and Margin-based Bounds
More details: General: http://www.learning-with-kernels.org/ Example of more complex bounds: http://www.research.ibm.com/people/t/tzhang/papers/jmlr02_cover.ps.gz PAC-learning, VC Dimension and Margin-based
BACKPROPAGATION. Neural network training optimization problem. Deriving backpropagation
BACKPROPAGATION Neural network training optimization problem min J(w) w The application of gradient descent to this problem is called backpropagation. Backpropagation is gradient descent applied to J(w)
Support Vector Machine II
Support Vector Machine II Jia-Bin Huang ECE-5424G / CS-5824 Virginia Tech Spring 2019 Administrative HW 1 due tonight HW 2 released. Online Scalable Learning Adaptive to Unknown Dynamics and Graphs Yanning
Nonlinear Classification
Nonlinear Classification INFO-4604, Applied Machine Learning University of Colorado Boulder October 5-10, 2017 Prof. Michael Paul Linear Classification Most classifiers we ve seen use linear functions
Statistical Pattern Recognition
Statistical Pattern Recognition Support Vector Machine (SVM) Hamid R. Rabiee Hadi Asheri, Jafar Muhammadi, Nima Pourdamghani Spring 2013 http://ce.sharif.edu/courses/91-92/2/ce725-1/ Agenda Introduction
Learning Methods for Linear Detectors
Intelligent Systems: Reasoning and Recognition James L. Crowley ENSIMAG 2 / MoSIG M1 Second Semester 2011/2012 Lesson 20 27 April 2012 Contents Learning Methods for Linear Detectors Learning Linear Detectors...2
Support Vector Machines. Introduction to Data Mining, 2 nd Edition by Tan, Steinbach, Karpatne, Kumar
Data Mining Support Vector Machines Introduction to Data Mining, 2 nd Edition by Tan, Steinbach, Karpatne, Kumar 02/03/2018 Introduction to Data Mining 1 Support Vector Machines Find a linear hyperplane
Notes on Discriminant Functions and Optimal Classification
Notes on Discriminant Functions and Optimal Classification Padhraic Smyth, Department of Computer Science University of California, Irvine c 2017 1 Discriminant Functions Consider a classification problem
Sparse vectors recap. ANLP Lecture 22 Lexical Semantics with Dense Vectors. Before density, another approach to normalisation.
ANLP Lecture 22 Lexical Semantics with Dense Vectors Henry S. Thompson Based on slides by Jurafsky & Martin, some via Dorota Glowacka 5 November 2018 Previous lectures: Sparse vectors recap How to represent
ANLP Lecture 22 Lexical Semantics with Dense Vectors
ANLP Lecture 22 Lexical Semantics with Dense Vectors Henry S. Thompson Based on slides by Jurafsky & Martin, some via Dorota Glowacka 5 November 2018 Henry S. Thompson ANLP Lecture 22 5 November 2018 Previous
(COM4513/6513) Week 1. Nikolaos Aletras ( Department of Computer Science University of Sheffield
Natural Language Processing (COM4513/6513) Week 1 Part II: Text classification with the perceptron Nikolaos Aletras (http://www.nikosaletras.com) n.aletras@sheffield.ac.uk Department of Computer Science
A Deep Interpretation of Classifier Chains
A Deep Interpretation of Classifier Chains Jesse Read and Jaakko Holmén http://users.ics.aalto.fi/{jesse,jhollmen}/ Aalto University School of Science, Department of Information and Computer Science and
CS 188: Artificial Intelligence Spring Announcements
CS 188: Artificial Intelligence Spring 2010 Lecture 22: Nearest Neighbors, Kernels 4/18/2011 Pieter Abbeel UC Berkeley Slides adapted from Dan Klein Announcements On-going: contest (optional and FUN!)
Characterization of Jet Charge at the LHC
Characterization of Jet Charge at the LHC Thomas Dylan Rueter, Krishna Soni Abstract The Large Hadron Collider (LHC) produces a staggering amount of data - about 30 petabytes annually. One of the largest
Faults Loading and Display in CE8R2 and R3. David Worsick, Calgary February 25, 2009
Faults Loading and Display in CE8R2 and R3 David Worsick, Calgary February 25, 2009 Faults Loading and Display Contents Faults in Hampson-Russell Normal Faults Strike-slip Faults Reverse Faults Thrust
Advanced Introduction to Machine Learning CMU-10715
Advanced Introduction to Machine Learning CMU-10715 Risk Minimization Barnabás Póczos What have we seen so far? Several classification & regression algorithms seem to work fine on training datasets: Linear
Optical Chinese Character Recognition using Probabilistic Neural Networks
Optical Chinese Character Recognition using Probabilistic Neural Networks Richard Romero, David Touretzky, and Robert Thibadeau Imaging Systems Lab, Robotics Institute Carnegie Mellon University Pittsburgh,
DETECTING HUMAN ACTIVITIES IN THE ARCTIC OCEAN BY CONSTRUCTING AND ANALYZING SUPER-RESOLUTION IMAGES FROM MODIS DATA INTRODUCTION
DETECTING HUMAN ACTIVITIES IN THE ARCTIC OCEAN BY CONSTRUCTING AND ANALYZING SUPER-RESOLUTION IMAGES FROM MODIS DATA Shizhi Chen and YingLi Tian Department of Electrical Engineering The City College of
Sound Recognition in Mixtures
Sound Recognition in Mixtures Juhan Nam, Gautham J. Mysore 2, and Paris Smaragdis 2,3 Center for Computer Research in Music and Acoustics, Stanford University, 2 Advanced Technology Labs, Adobe Systems
11 More Regression; Newton s Method; ROC Curves
More Regression; Newton s Method; ROC Curves 59 11 More Regression; Newton s Method; ROC Curves LEAST-SQUARES POLYNOMIAL REGRESSION Replace each X i with feature vector e.g. (X i ) = [X 2 i1 X i1 X i2
Support'Vector'Machines. Machine(Learning(Spring(2018 March(5(2018 Kasthuri Kannan
Support'Vector'Machines Machine(Learning(Spring(2018 March(5(2018 Kasthuri Kannan kasthuri.kannan@nyumc.org Overview Support Vector Machines for Classification Linear Discrimination Nonlinear Discrimination
Large-scale classification of traffic signs under real-world conditions
Large-scale classification of traffic signs under real-world conditions Lykele Hazelhoff a,b, Ivo Creusen a,b, Dennis van de Wouw a,b and Peter H.N. de With a,b a CycloMedia Technology B.V., Achterweg
CMU-Q Lecture 24:
CMU-Q 15-381 Lecture 24: Supervised Learning 2 Teacher: Gianni A. Di Caro SUPERVISED LEARNING Hypotheses space Hypothesis function Labeled Given Errors Performance criteria Given a collection of input
Foundations of Natural Language Processing Lecture 6 Spelling correction, edit distance, and EM
Foundations of Natural Language Processing Lecture 6 Spelling correction, edit distance, and EM Alex Lascarides (Slides from Alex Lascarides and Sharon Goldwater) 2 February 2019 Alex Lascarides FNLP Lecture
PATTERN CLASSIFICATION
PATTERN CLASSIFICATION Second Edition Richard O. Duda Peter E. Hart David G. Stork A Wiley-lnterscience Publication JOHN WILEY & SONS, INC. New York Chichester Weinheim Brisbane Singapore Toronto CONTENTS
Lecture 4: Feed Forward Neural Networks
Lecture 4: Feed Forward Neural Networks Dr. Roman V Belavkin Middlesex University BIS4435 Biological neurons and the brain A Model of A Single Neuron Neurons as data-driven models Neural Networks Training
Pattern Recognition and Machine Learning. Learning and Evaluation of Pattern Recognition Processes
Pattern Recognition and Machine Learning James L. Crowley ENSIMAG 3 - MMIS Fall Semester 2016 Lesson 1 5 October 2016 Learning and Evaluation of Pattern Recognition Processes Outline Notation...2 1. The
Samson Zhou. Pattern Matching over Noisy Data Streams
Samson Zhou Pattern Matching over Noisy Data Streams Finding Structure in Data Pattern Matching Finding all instances of a pattern within a string ABCD ABCAABCDAACAABCDBCABCDADDDEAEABCDA Knuth-Morris-Pratt
University of New Mexico Department of Computer Science. Final Examination. CS 561 Data Structures and Algorithms Fall, 2013
University of New Mexico Department of Computer Science Final Examination CS 561 Data Structures and Algorithms Fall, 2013 Name: Email: This exam lasts 2 hours. It is closed book and closed notes wing
A Discriminatively Trained, Multiscale, Deformable Part Model
A Discriminatively Trained, Multiscale, Deformable Part Model P. Felzenszwalb, D. McAllester, and D. Ramanan Edward Hsiao 16-721 Learning Based Methods in Vision February 16, 2009 Images taken from P.
A Decision Stump. Decision Trees, cont. Boosting. Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University. October 1 st, 2007
Decision Trees, cont. Boosting Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University October 1 st, 2007 1 A Decision Stump 2 1 The final tree 3 Basic Decision Tree Building Summarized