Data Analysis, Statistics, Machine Learning

Transcription

1 Data Analysis, Statistics, Machine Learning Leland Wilkinsn Adjunct Prfessr UIC Cmputer Science Chief Scien<st H2O.ai

2 Data Analysis What is data analysis? Summaries f batches f data Methds fr discvering pajerns in data Methds fr visualizing data Benefits Data analysis helps us supprt suppsi<ns Data analysis helps us discredit false explana<ns Data analysis helps us generate new ideas t inves<gate 2 Cpyright 2016 Leland Wilkinsn

3 Sta<s<cs What is (are) sta<s<cs? Summaries f samples frm ppula<ns Methds fr analyzing samples Making inferences based n samples Benefits Sta<s<cs help us avid false cnclusins when evalua<ng evidence Sta<s<cs prtect us frm being fled by randmness Sta<s<cs help us find pajerns in nnrandm events Sta<s<cs quan<fy risk Sta<s<cs cunteract ingrained bias in human judgment Sta<s<cal mdels are understandable by humans 3 Cpyright 2016 Leland Wilkinsn

4 Machine Learning What is machine learning? Data mining systems Discver pajerns in data Learning systems Benefits Adapt mdels ver <me ML helps t predict utcmes ML Xen utperfrms tradi<nal sta<s<cal predic<n methds ML mdels d nt need t be understd by humans Mst ML results are unintelligible (the excep<ns prve the rule) ML peple care abut the quality f a predic<n, nt the meaning f the result ML is ht (Deep Learning!, Big Data!) 4 Cpyright 2016 Leland Wilkinsn

5 Curse Outline 1. Intrduc<n 2. Data 3. Visualizing 4. Explring 5. Summarizing 6. Distribu<ns 7. Inference 8. Predic<ng 9. Smthing 10. Time Series 11. Cmparing 12. Reducing 13. Gruping 14. Learning 15. Anmalies 16. Analyzing 5 Cpyright 2016 Leland Wilkinsn

6 Data What is (are) data? A datum is a given (as in French dnnée) data is plural f datum Data may have many different frms Set, Bag, List, Table, etc. Many f these frms are amenable t data analysis Nne f these frms is suitable fr sta<s<cal analysis Sta<s<cs perate n variables, nt data A variable is a func<n mapping data bjects t values A randm variable is a variable whse values are each assciated with a prbability p (0 p 1) Visualiza<ns perate n data r variables 6 Cpyright 2016 Leland Wilkinsn

7 Visualizing Visualiza<ns represent data Tallies, stem- and- leaf plts, histgrams, pie charts, bar charts, Sta<s<cal visualiza<ns represent variables Prbability plts, density plts, Sta<s<cal visualiza<ns aid diagnsis f mdels Des a variable derive frm a given distribu<n? Are there utliers and ther anmalies? Are there trends (r peridicity, etc.) acrss <me? Are there rela<nships between variables? Are there clusters f pints (cases)? 7 Cpyright 2016 Leland Wilkinsn

8 Explring Explratry Data Analysis (Jhn W. Tukey, EDA) Summaries Transfrma<ns Smthing Rbustness Interac<vity What EDA is nt Lelng the data speak fr itself Fishing expedi<ns Null hypthesis tes<ng Qualita<ve Data Analysis Mixed methds Old wine in new bjles 8 Cpyright 2016 Leland Wilkinsn

9 Summarizing We summarize t remve irrelevant detail We summarize batches f data in a few numbers We summarize variables thrugh their distribu<ns The best summaries preserve imprtant infrma<n All summaries sacrifice infrma<n (lssy) Summaries Lca<n Ppular: mean, median, mde Others: weighted mean, trimmed mean, Spread Ppular: sd, range Others: Interquar<le Range, Median Abslute Devia<n, Shape Skewness Kurtsis 9 Cpyright 2016 Leland Wilkinsn

10 Distribu<ns A prbability func<n is a nnnega<ve func<n Its area (r mass) is 1 Distribu<ns are families f prbability func<ns Mst sta<s<cal methds depend n distribu<ns Nnparametric methds are distribu<n- free The Nrmal (Gaussian) distribu<n is mst ppular Other distribu<ns (Binmial, Pissn, ) are Xen used We use the Nrmal because f the Central Limit Therem Variables based n real data are rarely nrmally distributed But sums r means f randm variables tend t be S if we are drawing inferences abut means, Nrmal is usually OK This invlves a leap f faith 10 Cpyright 2016 Leland Wilkinsn

11 Inference Inference invlves drawing cnclusins frm evidence In lgic, the evidence is a set f premises In data analysis, the evidence is a set f data In sta<s<cs, the evidence is a sample frm a ppula<n A ppula<n is assumed t have a distribu<n The sample is assumed t be randm (Sme<mes there are ways arund that) The ppula<n may be the same size as the sample (nt usually a gd idea) There are tw histrical appraches t sta<s<cal inference Frequen<st Bayesian There are many widespread abuses f sta<s<cal inference We cherry pick ur results (scien<sts, jurnals, reprters, ) We didn t have a big enugh sample t detect a real difference We think a large sample guarantees accuracy (the bigger the bejer) 11 Cpyright 2016 Leland Wilkinsn

12 Predic<ng Mst sta<s<cal predic<n mdels take ne f tw frms y = Σ j (β j x j ) + ε (addi<ve func<n) y = f(x j, ε) (nnlinear func<n) The dis<nc<n is imprtant The first frm is called an addi<ve mdel The secnd frm is called a nnlinear mdel Addi<ve mdels can be curvilinear (if terms are nnlinear) Nnlinear mdels cannt be transfrmed t linear Examples f linear r linearizable mdels are y =β 0 + β 1 x β p x p + ε y =αe βx+ ε Examples f nnlinear mdels are y =β 1 x 1 / β 2 x 2 + ε y = lgβ 1 x 1 ε 12 Cpyright 2016 Leland Wilkinsn

13 Smthing Sme<mes we want t smth variables r rela<ns Tukey phrased this as data = smth + rugh The smthed versin shuld shw pajerns nt evident in raw data Many f these methds are nnparametric Sme are parametric But we use them t discver, nt t cnfirm 13 Cpyright 2016 Leland Wilkinsn

14 Time Series Time series sta<s<cs invlve randm prcesses ver <me Spa<al sta<s<cs invlve randm prcesses ver space Bth invlve similar mathema<cal mdels When there is n tempral r spa<al influence, these bil dwn t rdinary sta<s<cal methds DO NOT USE i.i.d. methds n tempral/spa<al data These require stchas<c mdels, nt trend lines measurements at each <me/space pint are nt independent 1.0 Autcrrelatin Plt Sales Year Quarterly US Ecmmerce Retail Sales, Seasnally Adjusted Crrelatin Lag Cpyright 2016 Leland Wilkinsn

15 Cmparing Sta<s<cal methds exist fr cmparing 2 r mre grups The classical apprach is Analysis f Variance (ANOVA) This methd invented by Sir Rnald Fisher It revlu<nized industrial/scien<fic experiments The researcher was able t examine mre than ne treatment at a <me With nly tw grups, results f Student s t- test and F- test are equivalent Mul<variate Analysis f Variance (MANOVA) This is ANOVA fr mre than ne dependent variable (utcme) Hierarchical mdeling is fr nested data There are several frms f this mul<level mdeling 15 Cpyright 2016 Leland Wilkinsn

16 Reducing Reducing takes many variables and reduces them t a smaller number f variables There are many ways t d this Principal cmpnents (PC) cnstructs rthgnal weighted cmpsites based n crrela<ns (cvariances) amng variables Mul<dimensinal Scaling (MDS) embeds them in a lw- dimensinal space based n distances between variables Manifld learning prjects them nt a lw- dimensinal nnlinear manifld Randm prjec<n is like principal cmpnents except the weights are randm. 16 Cpyright 2016 Leland Wilkinsn

17 Gruping We can create grups f variables r grups f cases These methds invlve what we call Cluster Analysis Hierarchical methds make trees f nested clusters Nn- hierarchical methds grup cases int k clusters These k clusters may be discrete r verlapping Tw cnsidera<ns are especially imprtant Distance/Dissimilarity measure Agglmera<n r splilng rule The cllec<n f clustering methds is huge Early applica<ns were fr numerical taxnmy in bilgy 17 Cpyright 2016 Leland Wilkinsn

18 Learning Machine Learning (ML) methds lk fr pajerns that persist acrss a large cllec<n f data bjects ML learns frm new data Key cncepts Curse f dimensinality Randm prjec<ns Regulariza<n Kernels Btstrap aggrega<n Bs<ng Ensembles Valida<n Methds Supervised Classifica<n (Discriminant Analysis, Supprt Vectr Machines, Trees, Set Cvers) Predic<n (Regressin, Trees, Neural Netwrks) Unsupervised Neural Netwrks Clustering Prjec<ns (PC, MDS, Manifld Learning) 18 Cpyright 2016 Leland Wilkinsn

19 Anmalies Anmalies are, literally, lack f a law (nms) The best- knwn anmaly is an utlier This presumes a distribu<n with tail(s) All utliers are anmalies, but nt all anmalies are utliers Iden<fying utliers is nt simple Almst every sxware system and sta<s<cs text gets it wrng Other anmalies dn t invlve distribu<ns Cding errrs in data Misspellings Singular events OXen anmalies in residuals are mre interes<ng than the es<mated values 19 Cpyright 2016 Leland Wilkinsn

20 Analyzing What Sta<s<cs is nt mathema<cs machine learning cmputer science prbability thery Sta<s<cal reasning is ra<nal Sta<s<cs cndi<ns cnclusins Sta<s<cs factrs ut randmness Wise wrds David Mre Stephen S<gler TFSI 20 Cpyright 2016 Leland Wilkinsn

21 References Sta<s<cs andrewgelman.cm statsblgs.cm jerrydallal.cm Visualiza<n flwingdata.cm eagereyes.rg Machine Learning hunch.net nlpers.blgspt.cm Math qumdcumque.wrdpress.cm terryta.wrdpress.cm 21 Cpyright 2016 Leland Wilkinsn

22 References Abelsn, R.P. (2005). Statistics as Principled Argument. Hillsdale, N.J.: L. Erlbaum. DeVeaux, R.D., Velleman, P., and Bck, D.E. (2013). Intr Stats (4 th Ed.). New Yrk: Pearsn. Freedman, D.A., Pisani, R. and Purves, R,A. (1978). Statistics. New Yrk: W.W. Nrtn. 22 Cpyright 2016 Leland Wilkinsn