Detection of Unauthorized Electricity Consumption using Machine Learning

Size: px
Start display at page:

Download "Detection of Unauthorized Electricity Consumption using Machine Learning"

Transcription

1 Detection of Unauthorized Electricity Consumption using Machine Learning Bo Tang, Ph.D. Department of Electrical and Computer Engineering Mississippi State University

2 Outline Advanced Metering Infrastructure (AMI) in Smart Grid Unauthorized Electricity Consumption (UEC) Detection of UEC using Machine Learning Algorithms Preliminary Simulation Results Future Work and Challenges

3 Overview of Advanced Metering Infrastructure AMI: Comprised of state of the art electronic/digital hardware and software. Enable detailed, time based data measurements and their transmissions. Benefits: System operation benefits Customer service benefits Financial benefits Source: Electric Power Research Institute

4 Unauthorized Electricity Consumption Ways of unauthorized electricity consumption (energy theft) Taking connections directly from distribution line Grounding the neutral wire Inserting some disc to stop rotating of the coil Hitting the meter to damage the rotating coil Interchanging input output connections Difficult to check these issues with AMI. It is estimated that utility companies lose more than $25 billion every year due to energy theft around the world.

5 Techniques for UEC Detection Statistical approaches Hypothesis test Learning based Approaches SVM, Neural Networks, Fuzzy classification, ARMA GLRT State based approaches Sensor monitoring, RFID monitoring, Mutual inspection, State estimation based. Game theory based approaches

6 Detection of UEC with Machine Learning Fig. 1 Basic procedure for learning based energy theft detection

7 Anomaly Detection Anomaly is a pattern that does not conform to the expected behavior. Also referred to outliers, exceptions, surprises, novelty, etc. General Steps for Anomaly Detection: Build a profile (or pattern) of the normal behavior Use the normal profile to detect anomalies Anomalies are observations whose characteristics differ significantly from normal profile.

8 Proposed Method Clustering the normal data to build multimodal profiles K mean clustering algorithms Silhouette (cohesion and separation) measure is to used to determine the number of patterns (clusters) Apply distance based (neighbors based) anomaly detection approaches. p

9 Relative Density-based Outlier Score (RDOS) Local Kernel Density Estimation

10 Relative Density-based Outlier Score (RDOS) Relative Density based Outlier Scores Anomalies detections p

11 RDOS: Theoretical Properties

12 Experimental Results Datasets: Smart energy data from the Irish Smart Energy Trial, including hourly electricity usage reports of Irish homes in 2009 and Synthetic unauthorized energy consumption Seven types of energy theft are generated.

13 Experimental Results

14 Experimental Results Seven distance based anomaly detection algorithms: Relative Density based Outlier Score (RDOS) Local Outlier Factor (LOF) Local Density Factor (LDF) Flexible Kernel Density Estimates (KDEOS) Influenced Outlierness (INFLO) Mutual k nearest neighbor (MNN) Indegree Number(ODIN)

15 Experimental Results Top 3 anomaly detection algorithms: AUC (area under ROC curve) Theft Type 1st 2nd 3rd 1 RDOS(0.88) LDF(0.84) INFLO(0.80) 2 LDF(0.79) RDOS(0.76) ODIN(0.72) 3 RDOS(0.93) MNN(0.86) ODIN(0.81) 4 RDOS(0.94) ODIN(0.86) INFLO(0.85) 5 RDOS(0.96) LDF(0.90) INFLO(0.88) 6 LDF(0.95) RDOS(0.93) ODIN(0.88) 7 RDOS(0.94) LDF(0.89) KDEOS(0.8) Example of ROC curve OA RDOS(0.92) LDF(0.85) INFLO(0.80)

16 Future Work Unauthorized energy consumption detection Advanced detection methods Real life data sets Privacy preservation in AMI Cloud computing and big data

17 Questions?

Evaluating Electricity Theft Detectors in Smart Grid Networks. Group 5:

Evaluating Electricity Theft Detectors in Smart Grid Networks. Group 5: Evaluating Electricity Theft Detectors in Smart Grid Networks Group 5: Ji, Xinyu Rivera, Asier Agenda Introduction Background Electricity Theft Detectors and Attacks Results Discussion Conclusion 2 Introduction

More information

The Abnormal Electricity Consumption Detection System Based on the Outlier Behavior Pattern Recognition

The Abnormal Electricity Consumption Detection System Based on the Outlier Behavior Pattern Recognition 2017 International Conference on Energy, Power and Environmental Engineering (ICEPEE 2017) ISBN: 978-1-60595-456-1 The Abnormal Electricity Consumption Detection System Based on the Outlier Behavior Pattern

More information

An Overview of Outlier Detection Techniques and Applications

An Overview of Outlier Detection Techniques and Applications Machine Learning Rhein-Neckar Meetup An Overview of Outlier Detection Techniques and Applications Ying Gu connygy@gmail.com 28.02.2016 Anomaly/Outlier Detection What are anomalies/outliers? The set of

More information

Anomaly (outlier) detection. Huiping Cao, Anomaly 1

Anomaly (outlier) detection. Huiping Cao, Anomaly 1 Anomaly (outlier) detection Huiping Cao, Anomaly 1 Outline General concepts What are outliers Types of outliers Causes of anomalies Challenges of outlier detection Outlier detection approaches Huiping

More information

Graph-Based Anomaly Detection with Soft Harmonic Functions

Graph-Based Anomaly Detection with Soft Harmonic Functions Graph-Based Anomaly Detection with Soft Harmonic Functions Michal Valko Advisor: Milos Hauskrecht Computer Science Department, University of Pittsburgh, Computer Science Day 2011, March 18 th, 2011. Anomaly

More information

CS570 Data Mining. Anomaly Detection. Li Xiong. Slide credits: Tan, Steinbach, Kumar Jiawei Han and Micheline Kamber.

CS570 Data Mining. Anomaly Detection. Li Xiong. Slide credits: Tan, Steinbach, Kumar Jiawei Han and Micheline Kamber. CS570 Data Mining Anomaly Detection Li Xiong Slide credits: Tan, Steinbach, Kumar Jiawei Han and Micheline Kamber April 3, 2011 1 Anomaly Detection Anomaly is a pattern in the data that does not conform

More information

Chart types and when to use them

Chart types and when to use them APPENDIX A Chart types and when to use them Pie chart Figure illustration of pie chart 2.3 % 4.5 % Browser Usage for April 2012 18.3 % 38.3 % Internet Explorer Firefox Chrome Safari Opera 35.8 % Pie chart

More information

Anomaly Detection. Jing Gao. SUNY Buffalo

Anomaly Detection. Jing Gao. SUNY Buffalo Anomaly Detection Jing Gao SUNY Buffalo 1 Anomaly Detection Anomalies the set of objects are considerably dissimilar from the remainder of the data occur relatively infrequently when they do occur, their

More information

Surprise Detection in Science Data Streams Kirk Borne Dept of Computational & Data Sciences George Mason University

Surprise Detection in Science Data Streams Kirk Borne Dept of Computational & Data Sciences George Mason University Surprise Detection in Science Data Streams Kirk Borne Dept of Computational & Data Sciences George Mason University kborne@gmu.edu, http://classweb.gmu.edu/kborne/ Outline Astroinformatics Example Application:

More information

Quantum machine learning for quantum anomaly detection CQT AND SUTD, SINGAPORE ARXIV:

Quantum machine learning for quantum anomaly detection CQT AND SUTD, SINGAPORE ARXIV: Quantum machine learning for quantum anomaly detection NANA LIU CQT AND SUTD, SINGAPORE ARXIV:1710.07405 TUESDAY 7 TH NOVEMBER 2017 QTML 2017, VERONA Types of anomaly detection algorithms Classification-based

More information

CSE 546 Final Exam, Autumn 2013

CSE 546 Final Exam, Autumn 2013 CSE 546 Final Exam, Autumn 0. Personal info: Name: Student ID: E-mail address:. There should be 5 numbered pages in this exam (including this cover sheet).. You can use any material you brought: any book,

More information

SUPERVISED LEARNING: INTRODUCTION TO CLASSIFICATION

SUPERVISED LEARNING: INTRODUCTION TO CLASSIFICATION SUPERVISED LEARNING: INTRODUCTION TO CLASSIFICATION 1 Outline Basic terminology Features Training and validation Model selection Error and loss measures Statistical comparison Evaluation measures 2 Terminology

More information

URD Cable Fault Prediction Model

URD Cable Fault Prediction Model 1 URD Cable Fault Prediction Model Christopher Gubala ComEd General Engineer Reliability Analysis 2014 IEEE PES General Meeting Utility Current Practices & Challenges of Predictive Distribution Reliability

More information

SYSTEMATIC CONSTRUCTION OF ANOMALY DETECTION BENCHMARKS FROM REAL DATA. Outlier Detection And Description Workshop 2013

SYSTEMATIC CONSTRUCTION OF ANOMALY DETECTION BENCHMARKS FROM REAL DATA. Outlier Detection And Description Workshop 2013 SYSTEMATIC CONSTRUCTION OF ANOMALY DETECTION BENCHMARKS FROM REAL DATA Outlier Detection And Description Workshop 2013 Authors Andrew Emmott emmott@eecs.oregonstate.edu Thomas Dietterich tgd@eecs.oregonstate.edu

More information

V e h i c l e I C T A r e n a I n n o v a t i o n B a z a a r L i n d h o l m e n S c i e n c e P a r k

V e h i c l e I C T A r e n a I n n o v a t i o n B a z a a r L i n d h o l m e n S c i e n c e P a r k I ndustrial Mathematics Applied to M a c hine Learning, B ig Data Analytics and Data Science V e h i c l e I C T A r e n a I n n o v a t i o n B a z a a r L i n d h o l m e n S c i e n c e P a r k 2018-02-

More information

HYPERGRAPH BASED SEMI-SUPERVISED LEARNING ALGORITHMS APPLIED TO SPEECH RECOGNITION PROBLEM: A NOVEL APPROACH

HYPERGRAPH BASED SEMI-SUPERVISED LEARNING ALGORITHMS APPLIED TO SPEECH RECOGNITION PROBLEM: A NOVEL APPROACH HYPERGRAPH BASED SEMI-SUPERVISED LEARNING ALGORITHMS APPLIED TO SPEECH RECOGNITION PROBLEM: A NOVEL APPROACH Hoang Trang 1, Tran Hoang Loc 1 1 Ho Chi Minh City University of Technology-VNU HCM, Ho Chi

More information

Holdout and Cross-Validation Methods Overfitting Avoidance

Holdout and Cross-Validation Methods Overfitting Avoidance Holdout and Cross-Validation Methods Overfitting Avoidance Decision Trees Reduce error pruning Cost-complexity pruning Neural Networks Early stopping Adjusting Regularizers via Cross-Validation Nearest

More information

Using Image Moment Invariants to Distinguish Classes of Geographical Shapes

Using Image Moment Invariants to Distinguish Classes of Geographical Shapes Using Image Moment Invariants to Distinguish Classes of Geographical Shapes J. F. Conley, I. J. Turton, M. N. Gahegan Pennsylvania State University Department of Geography 30 Walker Building University

More information

PAC-learning, VC Dimension and Margin-based Bounds

PAC-learning, VC Dimension and Margin-based Bounds More details: General: http://www.learning-with-kernels.org/ Example of more complex bounds: http://www.research.ibm.com/people/t/tzhang/papers/jmlr02_cover.ps.gz PAC-learning, VC Dimension and Margin-based

More information

9/26/17. Ridge regression. What our model needs to do. Ridge Regression: L2 penalty. Ridge coefficients. Ridge coefficients

9/26/17. Ridge regression. What our model needs to do. Ridge Regression: L2 penalty. Ridge coefficients. Ridge coefficients What our model needs to do regression Usually, we are not just trying to explain observed data We want to uncover meaningful trends And predict future observations Our questions then are Is β" a good estimate

More information

Be able to define the following terms and answer basic questions about them:

Be able to define the following terms and answer basic questions about them: CS440/ECE448 Section Q Fall 2017 Final Review Be able to define the following terms and answer basic questions about them: Probability o Random variables, axioms of probability o Joint, marginal, conditional

More information

Unsupervised Learning Methods

Unsupervised Learning Methods Structural Health Monitoring Using Statistical Pattern Recognition Unsupervised Learning Methods Keith Worden and Graeme Manson Presented by Keith Worden The Structural Health Monitoring Process 1. Operational

More information

Surprise Detection in Multivariate Astronomical Data Kirk Borne George Mason University

Surprise Detection in Multivariate Astronomical Data Kirk Borne George Mason University Surprise Detection in Multivariate Astronomical Data Kirk Borne George Mason University kborne@gmu.edu, http://classweb.gmu.edu/kborne/ Outline What is Surprise Detection? Example Application: The LSST

More information

An Introduction to Machine Learning

An Introduction to Machine Learning An Introduction to Machine Learning L2: Instance Based Estimation Alexander J. Smola Statistical Machine Learning Program Canberra, ACT 0200 Australia Alex.Smola@nicta.com.au Tata Institute, Pune, January

More information

About Nnergix +2, More than 2,5 GW forecasted. Forecasting in 5 countries. 4 predictive technologies. More than power facilities

About Nnergix +2, More than 2,5 GW forecasted. Forecasting in 5 countries. 4 predictive technologies. More than power facilities About Nnergix +2,5 5 4 +20.000 More than 2,5 GW forecasted Forecasting in 5 countries 4 predictive technologies More than 20.000 power facilities Nnergix s Timeline 2012 First Solar Photovoltaic energy

More information

Improving Efficiency of PV Systems Using Statistical Performance Monitoring

Improving Efficiency of PV Systems Using Statistical Performance Monitoring TASK 13: PERFORMANCE AND RELIABILITY OF PV SYSTEMS Improving Efficiency of PV Systems Using Statistical Performance Monitoring Mike Green (M.G.Lightning Ltd. (ISR)) Eyal Brill (Decision Makers Ltd. (ISR))

More information

FACTORIZATION MACHINES AS A TOOL FOR HEALTHCARE CASE STUDY ON TYPE 2 DIABETES DETECTION

FACTORIZATION MACHINES AS A TOOL FOR HEALTHCARE CASE STUDY ON TYPE 2 DIABETES DETECTION SunLab Enlighten the World FACTORIZATION MACHINES AS A TOOL FOR HEALTHCARE CASE STUDY ON TYPE 2 DIABETES DETECTION Ioakeim (Kimis) Perros and Jimeng Sun perros@gatech.edu, jsun@cc.gatech.edu COMPUTATIONAL

More information

P R O G N O S T I C S

P R O G N O S T I C S P R O G N O S T I C S THE KEY TO PREDICTIVE MAINTENANCE @senseyeio Me BEng Digital Systems Engineer Background in aerospace & defence and large scale wireless sensing Software Verification & Validation

More information

Guidelines for comparing boxplots

Guidelines for comparing boxplots Comparing Data Sets Project IMP I Name When using boxplots to compare two or more batches of data, it is usually best to compare individual features in a methodical way. You may find the following guidelines

More information

Electrical and Computer Engineering Department University of Waterloo Canada

Electrical and Computer Engineering Department University of Waterloo Canada Predicting a Biological Response of Molecules from Their Chemical Properties Using Diverse and Optimized Ensembles of Stochastic Gradient Boosting Machine By Tarek Abdunabi and Otman Basir Electrical and

More information

Machine learning for pervasive systems Classification in high-dimensional spaces

Machine learning for pervasive systems Classification in high-dimensional spaces Machine learning for pervasive systems Classification in high-dimensional spaces Department of Communications and Networking Aalto University, School of Electrical Engineering stephan.sigg@aalto.fi Version

More information

Available online at ScienceDirect. Procedia Engineering 119 (2015 ) 13 18

Available online at   ScienceDirect. Procedia Engineering 119 (2015 ) 13 18 Available online at www.sciencedirect.com ScienceDirect Procedia Engineering 119 (2015 ) 13 18 13th Computer Control for Water Industry Conference, CCWI 2015 Real-time burst detection in water distribution

More information

Discovering The World Of Chemistry

Discovering The World Of Chemistry Discovering The World Of Chemistry Dr. Dimitrios Tzalis IMI Open Info Day: Horizon 2020 - Health, demographic change and wellbeing 18th September 2015 Brussel Taros Chemicals GmbH & Co. KG Taros: Stability,

More information

Evaluation of Two Level Classifier for Predicting Compressor Failures in Heavy Duty Vehicles. Slawomir Nowaczyk SAIS 2017 workshop, May

Evaluation of Two Level Classifier for Predicting Compressor Failures in Heavy Duty Vehicles. Slawomir Nowaczyk SAIS 2017 workshop, May Evaluation of Two Level Classifier for Predicting Compressor Failures in Heavy Duty Vehicles Yuantao Fan, Pablo De Moral & Slawomir Nowaczyk SAIS 2017 workshop, 15-16 May Objective & Motivation Predictive

More information

The Changing Landscape of Land Administration

The Changing Landscape of Land Administration The Changing Landscape of Land Administration B r e n t J o n e s P E, PLS E s r i World s Largest Media Company No Journalists No Content Producers No Photographers World s Largest Hospitality Company

More information

A Design of Theft Detection Framework for Smart Grid Network

A Design of Theft Detection Framework for Smart Grid Network A Design of Theft Detection Framework for Smart Grid Network by Victor Zikun Xu A thesis presented to the University of Waterloo in fulfillment of the thesis requirement for the degree of Master of Mathematics

More information

Copernicus Data Driven Services for Regional & Local Government in Greece

Copernicus Data Driven Services for Regional & Local Government in Greece Workshop on Copernicus uptake by public authorities Gabriel Mavrellis CEO, Geospatial Enabling Technologies Copernicus Data Driven Services for Regional & Local Government in Greece The Company Copernicus

More information

Research on Feature Selection in Power User Identification

Research on Feature Selection in Power User Identification Mathematics and Computer Science 2018; 3(3): 67-76 http://www.sciencepublishinggroup.com/j/mcs doi: 10.11648/j.mcs.20180303.11 ISSN: 2575-6036 (Print); ISSN: 2575-6028 (Online) Research on Feature Selection

More information

Neural Networks. David Rosenberg. July 26, New York University. David Rosenberg (New York University) DS-GA 1003 July 26, / 35

Neural Networks. David Rosenberg. July 26, New York University. David Rosenberg (New York University) DS-GA 1003 July 26, / 35 Neural Networks David Rosenberg New York University July 26, 2017 David Rosenberg (New York University) DS-GA 1003 July 26, 2017 1 / 35 Neural Networks Overview Objectives What are neural networks? How

More information

Evaluating Electricity Theft Detectors in Smart Grid Networks

Evaluating Electricity Theft Detectors in Smart Grid Networks Evaluating Electricity Theft Detectors in Smart Grid Networks Daisuke Mashima 1 and Alvaro A. Cárdenas 2 1 Georgia Institute of Technology mashima@cc.gatech.edu 2 Fujitsu Laboratories of America alvaro.cardenas-mora@us.fujitsu.com

More information

From statistics to data science. BAE 815 (Fall 2017) Dr. Zifei Liu

From statistics to data science. BAE 815 (Fall 2017) Dr. Zifei Liu From statistics to data science BAE 815 (Fall 2017) Dr. Zifei Liu Zifeiliu@ksu.edu Why? How? What? How much? How many? Individual facts (quantities, characters, or symbols) The Data-Information-Knowledge-Wisdom

More information

FROM WATER LEAKS TO WINE GRAPES: A NEW OUTLOOK FOR IMAGERY ANALYSIS

FROM WATER LEAKS TO WINE GRAPES: A NEW OUTLOOK FOR IMAGERY ANALYSIS Place image here (10 x 3.5 ) FROM WATER LEAKS TO WINE GRAPES: A NEW OUTLOOK FOR IMAGERY ANALYSIS ENVI AS A FRAMEWORK FOR CLOUD SERVICES & DEEP LEARNING Presented By Gordon Sumerling on Behalf of Cherie

More information

A Bayesian Perspective on Residential Demand Response Using Smart Meter Data

A Bayesian Perspective on Residential Demand Response Using Smart Meter Data A Bayesian Perspective on Residential Demand Response Using Smart Meter Data Datong-Paul Zhou, Maximilian Balandat, and Claire Tomlin University of California, Berkeley [datong.zhou, balandat, tomlin]@eecs.berkeley.edu

More information

Support Vector Machines

Support Vector Machines Support Vector Machines Hypothesis Space variable size deterministic continuous parameters Learning Algorithm linear and quadratic programming eager batch SVMs combine three important ideas Apply optimization

More information

Introduction to Machine Learning Midterm Exam

Introduction to Machine Learning Midterm Exam 10-701 Introduction to Machine Learning Midterm Exam Instructors: Eric Xing, Ziv Bar-Joseph 17 November, 2015 There are 11 questions, for a total of 100 points. This exam is open book, open notes, but

More information

Deep Learning. Convolutional Neural Networks Applications

Deep Learning. Convolutional Neural Networks Applications Deep Learning Using a Convolutional Neural Network Dr. Ing. Morris Riedel Adjunct Associated Professor School of Engineering and Natural Sciences, University of Iceland Research Group Leader, Juelich Supercomputing

More information

Application of Artificial Neural Networks in Evaluation and Identification of Electrical Loss in Transformers According to the Energy Consumption

Application of Artificial Neural Networks in Evaluation and Identification of Electrical Loss in Transformers According to the Energy Consumption Application of Artificial Neural Networks in Evaluation and Identification of Electrical Loss in Transformers According to the Energy Consumption ANDRÉ NUNES DE SOUZA, JOSÉ ALFREDO C. ULSON, IVAN NUNES

More information

Text Mining. Dr. Yanjun Li. Associate Professor. Department of Computer and Information Sciences Fordham University

Text Mining. Dr. Yanjun Li. Associate Professor. Department of Computer and Information Sciences Fordham University Text Mining Dr. Yanjun Li Associate Professor Department of Computer and Information Sciences Fordham University Outline Introduction: Data Mining Part One: Text Mining Part Two: Preprocessing Text Data

More information

David John Gagne II, NCAR

David John Gagne II, NCAR The Performance Impacts of Machine Learning Design Choices for Gridded Solar Irradiance Forecasting Features work from Evaluating Statistical Learning Configurations for Gridded Solar Irradiance Forecasting,

More information

Visualizing Big Data on Maps: Emerging Tools and Techniques. Ilir Bejleri, Sanjay Ranka

Visualizing Big Data on Maps: Emerging Tools and Techniques. Ilir Bejleri, Sanjay Ranka Visualizing Big Data on Maps: Emerging Tools and Techniques Ilir Bejleri, Sanjay Ranka Topics Web GIS Visualization Big Data GIS Performance Maps in Data Visualization Platforms Next: Web GIS Visualization

More information

Stat 401XV Final Exam Spring 2017

Stat 401XV Final Exam Spring 2017 Stat 40XV Final Exam Spring 07 I have neither given nor received unauthorized assistance on this exam. Name Signed Date Name Printed ATTENTION! Incorrect numerical answers unaccompanied by supporting reasoning

More information

Machine Learning Basics

Machine Learning Basics Security and Fairness of Deep Learning Machine Learning Basics Anupam Datta CMU Spring 2019 Image Classification Image Classification Image classification pipeline Input: A training set of N images, each

More information

FINAL: CS 6375 (Machine Learning) Fall 2014

FINAL: CS 6375 (Machine Learning) Fall 2014 FINAL: CS 6375 (Machine Learning) Fall 2014 The exam is closed book. You are allowed a one-page cheat sheet. Answer the questions in the spaces provided on the question sheets. If you run out of room for

More information

Michigan State University, East Lansing, MI USA. Lansing, MI USA.

Michigan State University, East Lansing, MI USA. Lansing, MI USA. On-line Supporting Information for: Using Cost-Effective Targeting to Enhance the Efficiency of Conservation Investments in Payments for Ecosystem Services Xiaodong Chen1,*, Frank Lupi2, Andrés Viña1,

More information

Quantum Artificial Intelligence and Machine Learning: The Path to Enterprise Deployments. Randall Correll. +1 (703) Palo Alto, CA

Quantum Artificial Intelligence and Machine Learning: The Path to Enterprise Deployments. Randall Correll. +1 (703) Palo Alto, CA Quantum Artificial Intelligence and Machine : The Path to Enterprise Deployments Randall Correll randall.correll@qcware.com +1 (703) 867-2395 Palo Alto, CA 1 Bundled software and services Professional

More information

One-class Label Propagation Using Local Cone Based Similarity

One-class Label Propagation Using Local Cone Based Similarity One-class Label Propagation Using Local Based Similarity Takumi Kobayashi and Nobuyuki Otsu Abstract In this paper, we propose a novel method of label propagation for one-class learning. For binary (positive/negative)

More information

Probabilistic Machine Learning. Industrial AI Lab.

Probabilistic Machine Learning. Industrial AI Lab. Probabilistic Machine Learning Industrial AI Lab. Probabilistic Linear Regression Outline Probabilistic Classification Probabilistic Clustering Probabilistic Dimension Reduction 2 Probabilistic Linear

More information

This gives us an upper and lower bound that capture our population mean.

This gives us an upper and lower bound that capture our population mean. Confidence Intervals Critical Values Practice Problems 1 Estimation 1.1 Confidence Intervals Definition 1.1 Margin of error. The margin of error of a distribution is the amount of error we predict when

More information

Electrical Energy Modeling In Y2E2 Building Based On Distributed Sensors Information

Electrical Energy Modeling In Y2E2 Building Based On Distributed Sensors Information Electrical Energy Modeling In Y2E2 Building Based On Distributed Sensors Information Mahmoud Saadat Saman Ghili Introduction Close to 40% of the primary energy consumption in the U.S. comes from commercial

More information

Outline. Introduction to SpaceStat and ESTDA. ESTDA & SpaceStat. Learning Objectives. Space-Time Intelligence System. Space-Time Intelligence System

Outline. Introduction to SpaceStat and ESTDA. ESTDA & SpaceStat. Learning Objectives. Space-Time Intelligence System. Space-Time Intelligence System Outline I Data Preparation Introduction to SpaceStat and ESTDA II Introduction to ESTDA and SpaceStat III Introduction to time-dynamic regression ESTDA ESTDA & SpaceStat Learning Objectives Activities

More information

Sparse Gaussian Markov Random Field Mixtures for Anomaly Detection

Sparse Gaussian Markov Random Field Mixtures for Anomaly Detection Sparse Gaussian Markov Random Field Mixtures for Anomaly Detection Tsuyoshi Idé ( Ide-san ), Ankush Khandelwal*, Jayant Kalagnanam IBM Research, T. J. Watson Research Center (*Currently with University

More information

Course in Data Science

Course in Data Science Course in Data Science About the Course: In this course you will get an introduction to the main tools and ideas which are required for Data Scientist/Business Analyst/Data Analyst. The course gives an

More information

Hurricane Prediction with Python

Hurricane Prediction with Python Hurricane Prediction with Python Minwoo Lee lemin@cs.colostate.edu Computer Science Department Colorado State University 1 Hurricane Prediction Hurricane Katrina We lost everything. Katrina didn t care

More information

Cost-Benefit Analysis of the Pooled- Fund Maintenance Decision Support System: Case Study

Cost-Benefit Analysis of the Pooled- Fund Maintenance Decision Support System: Case Study Cost-Benefit Analysis of the Pooled- Fund Maintenance Decision Support System: Case Study Zhirui Ye (WTI) Xianming Shi (WTI) Christopher K. Strong (City of Oshkosh) 12 th AASHTO-TRB TRB Maintenance Management

More information

Learning Theory Continued

Learning Theory Continued Learning Theory Continued Machine Learning CSE446 Carlos Guestrin University of Washington May 13, 2013 1 A simple setting n Classification N data points Finite number of possible hypothesis (e.g., dec.

More information

OVER 1.2 billion people do not have access to electricity

OVER 1.2 billion people do not have access to electricity 1 1 2 Predicting Low Voltage Events on Rural Micro-Grids in Tanzania Samuel Steyer, Shea Hughes, Natasha Whitney Abstract Our team initially set out to predict when and where low voltage events and subsequent

More information

Roberto Perdisci^+, Guofei Gu^, Wenke Lee^ presented by Roberto Perdisci. ^Georgia Institute of Technology, Atlanta, GA, USA

Roberto Perdisci^+, Guofei Gu^, Wenke Lee^ presented by Roberto Perdisci. ^Georgia Institute of Technology, Atlanta, GA, USA U s i n g a n E n s e m b l e o f O n e - C l a s s S V M C l a s s i f i e r s t o H a r d e n P a y l o a d - B a s e d A n o m a l y D e t e c t i o n S y s t e m s Roberto Perdisci^+, Guofei Gu^, Wenke

More information

FIND A FUNCTION TO CLASSIFY HIGH VALUE CUSTOMERS

FIND A FUNCTION TO CLASSIFY HIGH VALUE CUSTOMERS LINEAR CLASSIFIER 1 FIND A FUNCTION TO CLASSIFY HIGH VALUE CUSTOMERS x f y High Value Customers Salary Task: Find Nb Orders 150 70 300 100 200 80 120 100 Low Value Customers Salary Nb Orders 40 80 220

More information

Identification of False Data Injection Attacks with Considering the Impact of Wind Generation and Topology Reconfigurations

Identification of False Data Injection Attacks with Considering the Impact of Wind Generation and Topology Reconfigurations 1 Identification of False Data ion Attacks with Considering the Impact of Wind Generation and Topology Reconfigurations Mostafa Mohammadpourfard, Student Member, IEEE, Ashkan Sami, Member, IEEE, and Yang

More information

PAC-learning, VC Dimension and Margin-based Bounds

PAC-learning, VC Dimension and Margin-based Bounds More details: General: http://www.learning-with-kernels.org/ Example of more complex bounds: http://www.research.ibm.com/people/t/tzhang/papers/jmlr02_cover.ps.gz PAC-learning, VC Dimension and Margin-based

More information

Anomaly Detection via Over-sampling Principal Component Analysis

Anomaly Detection via Over-sampling Principal Component Analysis Anomaly Detection via Over-sampling Principal Component Analysis Yi-Ren Yeh, Zheng-Yi Lee, and Yuh-Jye Lee Abstract Outlier detection is an important issue in data mining and has been studied in different

More information

Beyond the Point Cloud: From Transductive to Semi-Supervised Learning

Beyond the Point Cloud: From Transductive to Semi-Supervised Learning Beyond the Point Cloud: From Transductive to Semi-Supervised Learning Vikas Sindhwani, Partha Niyogi, Mikhail Belkin Andrew B. Goldberg goldberg@cs.wisc.edu Department of Computer Sciences University of

More information

Tennis player segmentation for semantic behavior analysis

Tennis player segmentation for semantic behavior analysis Proposta di Tennis player segmentation for semantic behavior analysis Architettura Software per Robot Mobili Vito Renò, Nicola Mosca, Massimiliano Nitti, Tiziana D Orazio, Donato Campagnoli, Andrea Prati,

More information

Is Your GIS Ready For Grid Modernization? A State-of-the Industry Report

Is Your GIS Ready For Grid Modernization? A State-of-the Industry Report Is Your GIS Ready For Grid Modernization? A State-of-the Industry Report Is Your GIS Ready For Grid Modernization? Survey Overview In the first quarter of 2019, Energy Acuity conducted a grid modernization

More information

QualiMET 2.0. The new Quality Control System of Deutscher Wetterdienst

QualiMET 2.0. The new Quality Control System of Deutscher Wetterdienst QualiMET 2.0 The new Quality Control System of Deutscher Wetterdienst Reinhard Spengler Deutscher Wetterdienst Department Observing Networks and Data Quality Assurance of Meteorological Data Michendorfer

More information

A Novel Activity Detection Method

A Novel Activity Detection Method A Novel Activity Detection Method Gismy George P.G. Student, Department of ECE, Ilahia College of,muvattupuzha, Kerala, India ABSTRACT: This paper presents an approach for activity state recognition of

More information

Standardization of Quantum Cryptography in China

Standardization of Quantum Cryptography in China Standardization of Quantum Cryptography in China Zhengfu Han University of Science and Technology of China Anhui Asky Quantum Technology Co.,Ltd November 7,2018 CONTENTS 1 Background on Quantum Cryptography

More information

A Broad View of Geospatial Technology & Systems

A Broad View of Geospatial Technology & Systems A Broad View of Geospatial Technology & Systems Pete Large Vice President, Trimble On the shoulders of giants 1 Since their time, our ability to generate geospatial information has grown exponentially

More information

Predicting freeway traffic in the Bay Area

Predicting freeway traffic in the Bay Area Predicting freeway traffic in the Bay Area Jacob Baldwin Email: jtb5np@stanford.edu Chen-Hsuan Sun Email: chsun@stanford.edu Ya-Ting Wang Email: yatingw@stanford.edu Abstract The hourly occupancy rate

More information

Evolutionary Functional Link Interval Type-2 Fuzzy Neural System for Exchange Rate Prediction

Evolutionary Functional Link Interval Type-2 Fuzzy Neural System for Exchange Rate Prediction Evolutionary Functional Link Interval Type-2 Fuzzy Neural System for Exchange Rate Prediction 3. Introduction Currency exchange rate is an important element in international finance. It is one of the chaotic,

More information

Machine Learning for Computational Advertising

Machine Learning for Computational Advertising Machine Learning for Computational Advertising L1: Basics and Probability Theory Alexander J. Smola Yahoo! Labs Santa Clara, CA 95051 alex@smola.org UC Santa Cruz, April 2009 Alexander J. Smola: Machine

More information

Use of graphene by an energy utility

Use of graphene by an energy utility Use of graphene by an energy utility ENGIE A global player in the energy business (2015) Power Natural gas Energy services No.1 Independent Power Producer (IPP) in the world. No.1 producer of nonnuclear

More information

Style-aware Mid-level Representation for Discovering Visual Connections in Space and Time

Style-aware Mid-level Representation for Discovering Visual Connections in Space and Time Style-aware Mid-level Representation for Discovering Visual Connections in Space and Time Experiment presentation for CS3710:Visual Recognition Presenter: Zitao Liu University of Pittsburgh ztliu@cs.pitt.edu

More information

Multisurface Proximal Support Vector Machine Classification via Generalized Eigenvalues

Multisurface Proximal Support Vector Machine Classification via Generalized Eigenvalues Multisurface Proximal Support Vector Machine Classification via Generalized Eigenvalues O. L. Mangasarian and E. W. Wild Presented by: Jun Fang Multisurface Proximal Support Vector Machine Classification

More information

Explaining Machine Learning Decisions

Explaining Machine Learning Decisions Explaining Machine Learning Decisions Grégoire Montavon, TU Berlin Joint work with: Wojciech Samek, Klaus-Robert Müller, Sebastian Lapuschkin, Alexander Binder 18/09/2018 Intl. Workshop ML & AI, Telecom

More information

TDT4173 Machine Learning

TDT4173 Machine Learning TDT4173 Machine Learning Lecture 9 Learning Classifiers: Bagging & Boosting Norwegian University of Science and Technology Helge Langseth IT-VEST 310 helgel@idi.ntnu.no 1 TDT4173 Machine Learning Outline

More information

Article from. Predictive Analytics and Futurism. July 2016 Issue 13

Article from. Predictive Analytics and Futurism. July 2016 Issue 13 Article from Predictive Analytics and Futurism July 2016 Issue 13 Regression and Classification: A Deeper Look By Jeff Heaton Classification and regression are the two most common forms of models fitted

More information

LAB 5: Induction: A Linear Generator

LAB 5: Induction: A Linear Generator 1 Name Date Partner(s) OBJECTIVES LAB 5: Induction: A Linear Generator To understand how a changing magnetic field induces an electric field. To observe the effect of induction by measuring the generated

More information

3.4 Fuzzy Logic Fuzzy Set Theory Approximate Reasoning Fuzzy Inference Evolutionary Optimization...

3.4 Fuzzy Logic Fuzzy Set Theory Approximate Reasoning Fuzzy Inference Evolutionary Optimization... Contents 1 Introduction... 1 1.1 The Shale Revolution... 2 1.2 Traditional Modeling... 4 1.3 A Paradigm Shift... 4 2 Modeling Production from Shale... 7 2.1 Reservoir Modeling of Shale... 9 2.2 System

More information

Multi-Plant Photovoltaic Energy Forecasting Challenge: Second place solution

Multi-Plant Photovoltaic Energy Forecasting Challenge: Second place solution Multi-Plant Photovoltaic Energy Forecasting Challenge: Second place solution Clément Gautrais 1, Yann Dauxais 1, and Maël Guilleme 2 1 University of Rennes 1/Inria Rennes clement.gautrais@irisa.fr 2 Energiency/University

More information

FRaC: A Feature-Modeling Approach for Semi-Supervised and Unsupervised Anomaly Detection

FRaC: A Feature-Modeling Approach for Semi-Supervised and Unsupervised Anomaly Detection Noname manuscript No. (will be inserted by the editor) FRaC: A Feature-Modeling Approach for Semi-Supervised and Unsupervised Anomaly Detection Keith Noto Carla Brodley Donna Slonim Received: date / Accepted:

More information

WeatherCloud Hyper-Local Global Forecasting All rights reserved. Fathym, Inc.

WeatherCloud Hyper-Local Global Forecasting All rights reserved. Fathym, Inc. WeatherCloud Hyper-Local Global Forecasting based on current forecast techniques EVOLVING FORECASTING TECHNOLOGY 1) The WeatherCloud backend forecast system allows for routing around hazardous weather

More information

Machine Learning. Lecture 9: Learning Theory. Feng Li.

Machine Learning. Lecture 9: Learning Theory. Feng Li. Machine Learning Lecture 9: Learning Theory Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 2018 Why Learning Theory How can we tell

More information

VBM683 Machine Learning

VBM683 Machine Learning VBM683 Machine Learning Pinar Duygulu Slides are adapted from Dhruv Batra Bias is the algorithm's tendency to consistently learn the wrong thing by not taking into account all the information in the data

More information

Naveed Anwar, AIT Solutions

Naveed Anwar, AIT Solutions 1 Current Trends in Technologies How computing hardware and Software will impact the future Naveed Anwar, Ph.D. Executive Director, AIT Solutions 2 Computing and Technologies Software Hardware Programming

More information

Classification and Pattern Recognition

Classification and Pattern Recognition Classification and Pattern Recognition Léon Bottou NEC Labs America COS 424 2/23/2010 The machine learning mix and match Goals Representation Capacity Control Operational Considerations Computational Considerations

More information

Inexact Search is Good Enough

Inexact Search is Good Enough Inexact Search is Good Enough Advanced Machine Learning for NLP Jordan Boyd-Graber MATHEMATICAL TREATMENT Advanced Machine Learning for NLP Boyd-Graber Inexact Search is Good Enough 1 of 1 Preliminaries:

More information

ECML PKDD Discovery Challenges 2017

ECML PKDD Discovery Challenges 2017 ECML PKDD Discovery Challenges 2017 Roberto Corizzo 1 and Dino Ienco 2 1 Department of Computer Science, University of Bari Aldo Moro, Bari, Italy roberto.corizzo@uniba.it 2 Irstea, UMR TETIS, Univ. Montpellier,

More information

18.9 SUPPORT VECTOR MACHINES

18.9 SUPPORT VECTOR MACHINES 744 Chapter 8. Learning from Examples is the fact that each regression problem will be easier to solve, because it involves only the examples with nonzero weight the examples whose kernels overlap the

More information

Future Proofing the Provision of Geoinformation: Emerging Technologies

Future Proofing the Provision of Geoinformation: Emerging Technologies Future Proofing the Provision of Geoinformation: Emerging Technologies An Exchange Forum with the Geospatial Industry William Cartwright Chair JBGIS Second High Level Forum on Global Geospatial Information

More information