CONTEMPORARY ANALYTICAL ECOSYSTEM PATRICK HALL, SAS INSTITUTE

Size: px

Start display at page:

Download "CONTEMPORARY ANALYTICAL ECOSYSTEM PATRICK HALL, SAS INSTITUTE"

Jonathan McBride
6 years ago
Views:

2 Agenda (Optional) History Lesson 2015 Buzzwords Machine Learning for X Citizen Data Scientist High Quality, Distributed, Open-Source Analytics Closing C o p yr i g ht 2014, S A S I nstitute Inc. A l l r i g hts r eser v ed.

World s Data in Zettabytes Data growth 50.00 45.00 40.00 35.00 30.

3 World s Data in Zettabytes Data growth

4 Data growth (1 zettabyte = 1 billion terabytes)

5 In 2008 In 2013 Typical server hard drive was 500GB with a transfer rate of 98 MB/sec Typical Server Hard Drive was 4TB with a transfer rate of 150 MB/sec An entire Disk could be transferred in 85 minutes An entire disk could be transferred in 440 minutes

6 $1.20 Average Price 1MB RAM $1.00 $0.80 $0.60 $0.40 $0.20 $

7 4000 CPU Speed in MHz

8 Disk capacities are getting bigger, but disks are not spinning faster Processors are not running much faster, but they have more cores RAM is becoming affordable

9 So To handle all of this new data we distribute it on clusters of computers Most modern analytical architectures take advantage of in-memory, distributed processing

10 Yesterday s state-of-the-art: Multicore CPU GPU Solid state drive (SSD) Analyst Workstation Data Software Server Analyst Software Client Data MPI Based

disk-enabled SAS High-Performance Analytics or SAS LASR Analytic Server, in-memory Spark MLlib,

11 Today s state-of-the-art: Distributed storage platform Hadoop Distributed File System (HDFS) Massively parallel (MPP) databases Distributed analytics platform Hadoop MapReduce, disk-enabled SAS High-Performance Analytics or SAS LASR Analytic Server, in-memory Spark MLlib, H2O.ai, in-memory Data Scientist Software Client Distributed Data and Software on Multiple Servers MPI Based

12 Agenda (Optional) History Lesson 2015 Buzzwords Machine Learning for X Citizen Data Scientist High Quality, Distributed, Open-Source Analytics Closing

13 Buzzwords IoT Cloud You are here. Peak of Inflated Expectations Plateau of Productivity Trigger Trough of Disillusionment C o p yr i g ht 2014, S A S I nstitute Inc. A l l r i g hts r eser v ed.

14 Buzzwords: Internet of Things Sensor Data? Streaming Analytics? Privacy? C o p yr i g ht 2014, S A S I nstitute Inc. A l l r i g hts r eser v ed.

Data In Machine Learning Insights Out Security?

15 Buzzwords: Cloud Machine Learning Platforms Data Visualization Recommendation Image Recognition Data In Machine Learning Insights Out Security?? Copyright 2013, SAS Institute Inc. All rights reserved.

16 Buzzwords Big Data You are here. Peak of Inflated Expectations Plateau of Productivity Trigger Trough of Disillusionment C o p yr i g ht 2014, S A S I nstitute Inc. A l l r i g hts r eser v ed.

17 Hadoop Corporate Adoption Remains Low Death of RDBMS exaggerated Big data adoption will require time C o p yr i g ht 2014, S A S I nstitute Inc. A l l r i g hts r eser v ed.

18 Agenda (Optional) History Lesson 2015 Buzzwords Machine Learning for X Citizen Data Scientist High Quality, Distributed, Open-Source Analytics Closing C o p yr i g ht 2014, S A S I nstitute Inc. A l l r i g hts r eser v ed.

19 Statistics Pattern Recognition Computational Neuroscience Data Science Data Mining Machine Learning AI Databases KDD

20 Data Mining Machine Learning SUPERVISED LEARNING UNSUPERVISED LEARNING SEMI-SUPERVISED LEARNING TRANSDUCTION Regression LASSO regression Logistic regression Ridge regression Decision tree Gradient boosting Random forests Neural Know networks y SVM Naïve Bayes Neighbors Gaussian processes A priori rules Clustering k-means clustering Mean shift clustering Spectral clustering Kernel density estimation Don t Nonnegative matrix know y factorization PCA Kernel PCA Sparse PCA Singular value decomposition SOM Prediction and classification* Clustering* EM TSVM Manifold Sometimes regularization know y Autoencoders Multilayer perceptron Restricted Boltzmann machines REINFORCEMENT LEARNING EVOLUTIONARY LEARNING *In semi-supervised learning, supervised prediction and classification algorithms are often combined with clustering.

21 Machine Learning for X Let X = { Healthcare, Asset Protection, Manufacturing, Energy, Government, Security, Text Mining, } There is a desire to apply advances in machine learning more broadly. C o p yr i g ht 2014, S A S I nstitute Inc. A l l r i g hts r eser v ed.

22 Let X = Healthcare Ensemble models for epidemiology Predicting hospital readmission Looking forward: Electronic Medical Records (EMR) C o p yr i g ht 2014, S A S I nstitute Inc. A l l r i g hts r eser v ed.

23 Let X = Asset Protection A = QΛQ 1 Λ ii = λ i λ 2 > t C o p yr i g ht 2014, S A S I nstitute Inc. A l l r i g hts r eser v ed.

24 Let X = Manufacturing Quality Control Deep Learning C o p yr i g ht 2014, S A S I nstitute Inc. A l l r i g hts r eser v ed.

25 What is deep learning? Using a neural network with more than two hidden layers for a supervised or unsupervised learning task

26 Why is deep important? Multiple Scales:

27 Unsupervised training Target vector Input vectors y x 1 x 2 x

28 Unsupervised training Supervised Neural Network Unsupervised Neural Network y x 1 x 2 x 3 h 11 h 12 h 11 h 12 x 1 x 2 x 3 x 1 x 2 x 3 (Known as an autoencoder)

Unsupervised training and stacked layers y h 21 hy 22 h 23 h 31 h 32 h 21 h 22 h 23 h 11 h 12 h 13 h 14 h 11 h 12 13 h 14 31 h 32 The weights from layerwise pre-training can be used as an

29 Unsupervised training and stacked layers y h 21 hy 22 h 23 h 31 h 32 h 21 h 22 h 23 h 11 h 12 h 13 h 14 h 11 h h h 32 The weights from layerwise pre-training can be used as an initialization for training the entire deep network! x 1 h x 21 2 h x 3 x 4 x 22 h 23 5 h 11 h 12 h 13 h 14 x 1 x 2 x 3 x 4 x 5 x 1 x 2 x 3 x 4 x 5 Many separate, unsupervised, single hidden-layer networks are used to initialize a larger supervised network in a layerwise fashion

30 Let X = Government Shadow

31 Let X = Security

32 Target Layer Extractable Features Input Layer

34 PROC CLUSTER data=face.sas_gezichten4 method=average plots=all outtree = face.gezichtentree ccc pseudo; id name; RUN;

36 Let X = Energy Production

37 Uncorrupted Output Features Target Layer h5 h4 h3 h2 h1 Hidden Neurons Hidden Neurons Hidden Neurons Hidden Neurons Hidden Neurons Partially Corrupted Input Features Extractable Features Input Layer

38 Target 1 Target 2 Target 3 Target 4 W51 W52 W53 W54 h51 h52

39 h51 Edge Weights Handwritten Eight

41 Agenda (Optional) History Lesson 2015 Buzzwords Machine Learning for X Citizen Data Scientist High Quality, Distributed, Open-Source Analytics Closing

42 Citizen Data Scientist People using data for the good of humanity! because their boss said so. (grumble, grumble)

43 Citizen Data Scientist Datakind Watson Analytics Beyondcore

44 Agenda (Optional) History Lesson 2015 Buzzwords Machine Learning for X Citizen Data Scientist High Quality, Distributed, Open-Source Analytics Closing

45 High Quality, Distributed, Open-Source Analytics

46 High Quality, Distributed, Open-Source Analytics Excellent Core, ETL, and basic analytics RDD Dataframe MLlib is immature: Production analytics?

47 High Quality, Distributed, Open-Source Analytics Innovative data compression routines Full-featured analytics

48 Agenda (Optional) History Lesson 2015 Buzzwords Machine Learning for X Citizen Data Scientist High Quality, Distributed, Open-Source Analytics Closing

49 Don t be afraid yet

50 !!!???!!! I just built 850 new models. When can you put them into production? The IT folks The Analytics folks

Where you can find me DC Data Community Meetup - July 24

Production Analytics SAS Data Mining Community

51 Where you can find me DC Data Community Meetup - July 24 th Playing Nice: Using PMML, Python, R, and SAS for Production Analytics SAS Data Mining Community Quora Github Twitter github.com/jphall663

Machine learning comes from Bayesian decision theory in statistics. There we want to minimize the expected value of the loss function.

Bayesian learning: Machine learning comes from Bayesian decision theory in statistics. There we want to minimize the expected value of the loss function. Let y be the true label and y be the predicted