Exploratory Data Analysis

Transcription

1 CS448B :: 30 Sep 2010 Exploratory Data Analysis Last Time: Visualization Re-Design Jeffrey Heer Stanford University In-Class Design Exercise Mackinlay s Ranking Task: Analyze and Re-design visualization Identify data variables (n,o,q) and encodings Critique the design: what works, what doesn t Sketch a re-design to improve communication Present result to the class Conjectured effectiveness of the encoding 1

2 Re-Design Presentations 1. Describe the data and visualization (~1 min) 2. Present your critique 3. Briefly describe your re-design ideas Each group member should introduce themselves: name, department, etc Source: The Atlantic 300 no. 2 (September 2007) Number of Classified U.S. Documents 2

3 Washington Dulles Airport Map Source: United Airlines Hemispheres Source: National Geographic, September, 2008, p. 22. Silver, Mark. "High School Give-and-Take." 3

4 Source: Business Week, June 18,

5 Source: India Today 5

6 Preparing for a Pandemic Source: Scientific American, 293(5). November, 2005, p. 50 Source: Wired Magazine, September 2008 Music: Super Cuts (page 92) 6

7 Assignment 2: Exploratory Data Analysis Use visualization software (Tableau) to form & answer questions First steps: Step 1: Pick domain & data Step 2: Pose questions Step 3: Profile the data Iterate as needed Create visualizations Interact with data Refine your questions Make wiki notebook Keep record of your analysis Prepare a final graphic and caption Due by end-of-day Tuesday, October 12 Today: Exploratory Data Analysis The Future of Data Analysis, John W. Tukey

8 Set A Set B Set C Set D X Y X Y X Y X Y Y Y Set A Set C Set B Set D Summary Statistics Linear Regression u X = 9.0 σ X = Y 2 = X u Y = 7.5 σ Y = 2.03 R 2 = 0.67 Anscombe X X Topics Exploratory Data Analysis Data Diagnostics Graphical Methods Data Transformation Confirmatory Data Analysis Statistical Hypothesis Testing Placing the Two in Concert Data Diagnostics 8

9 How to gauge the quality of a visualization? The first sign that a visualization is good is that it shows you a problem in your data every successful visualization that I've been involved with has had this stage where you realize, "Oh my God, this data is not what I thought it would be!" So already, you've discovered something. - Martin Wattenberg Node-link Matrix 9

10 Matrix Visualize Friends by School? Berkeley Cornell Harvard Harvard University Stanford Stanford University UC Berkeley UC Davis University of California at Berkeley University of California, Berkeley University of California, Davis Data Quality & Usability Hurdles Missing Data Erroneous Values Type Conversion Entity Resolution Data Integration no measurements, redacted,? misspelling, outliers,? e.g., zip code to lat-lon diff. values for the same thing? effort/errors when combining data Exploratory Analysis: Effectiveness of Antibiotics LESSON: Anticipate problems with your data. Many research problems around these issues! 10

11 The Data Set What questions might we ask? Genus of Bacteria Species of Bacteria Antibiotic Applied Gram-Staining? Min. Inhibitory Concent. (g) String String String Pos / Neg Number Collected prior to Will Burtin, 1951 How do the drugs compare? Mike Bostock, CS448B Winter

12 How do the bacteria group with respect to antibiotic resistance? Not a streptococcus! (realized ~30 yrs later) Really a streptococcus! (realized ~20 yrs later) Bowen Li, CS448B Fall 2009 Wainer & Lysen American Scientist, 2009 Lessons Exploratory Process 1 Construct graphics to address questions 2 Inspect answer and assess new questions 3 Repeat! Transform the data appropriately (e.g., invert, log) How do the bacteria group w.r.t. resistance? Do different drugs correlate? Wainer & Lysen American Scientist, 2009 Show data variation, not design variation -Tufte 12

13 Common Data Transformations Normalize y i / Σ i y i (among others) Log log y Power y 1/k Box-Cox Transform (y λ 1) / λ if λ 0 log y if λ = 0 Binning e.g., histograms Grouping e.g., merge categories Exploratory Analysis: Participation on Amazon s Mechanical Turk Often performed to aid comparison (% or scale difference) or better approx. normal distribution The Data Set (~200 rows) Turker ID String Avg. Completion Rate Number [0,1] Min Median Lower Quartile Upper Quartile Max Collected in 2009 by Heer & Bostock. What questions might we ask of the data? What charts might provide insight? Turker Completion Percentage Box (and Whiskers) Plot 13

14 Turker Completion Percentage Turker Completion Percentage Dot Plot (with transparency to indicate overlap) Dot Plot w/ Reference Lines Turker Completion Percentage Histogram (binned counts) Stem-and-Leaf Plot 14

15 Quantile-Quantile Plot Used to compare two distributions; in this case, one actual and one theoretical. Plots the quantiles (here, the percentile values) against each other. Similar distributions lie along the diagonal. If linearly related, values will lie along a line, but with potentially varying slope and intercept. Quantile-Quantile Plots Lessons Even for simple data, a variety of graphics might provide insight. Again, tailor the choice of graphic to the questions being asked, but be open to surprises. Turker Completion Percentage Histogram + Fitted Mixture of 3 Gaussians Graphics can be used to understand and help assess the quality of statistical models. Premature commitment to a model and lack of verification can lead an analysis astray. 15

16 Some Uses of Formal Statistics Confirmatory Data Analysis What is the probability that the pattern I'm seeing might have arisen by chance? With what parameters does the data best fit a given function? What is the goodness of fit? How well do one (or more) data variables predict another? and many others. Example: Heights by Gender Gender Height (in) Male / Female Number µ m = 69.4 σ m = 4.69 N m = 1000 µ f = 63.8 σ f = 4.18 N f = 1000 Is this difference in heights significant? In other words: assuming no true difference, what is the prob. that our data is due to chance? Histograms 16

17 Bihistogram Formulating a Hypothesis Null Hypothesis (H 0 ): Alternate Hypothesis (H a ): µ m = µ f (population) µ m µ f (population) A statistical hypothesis test assesses the likelihood of the null hypothesis. What is the probability of sampling the observed data assuming population means are equal? This is called the p value. 17

18 Testing Procedure Compute test statistic Compute a test statistic. This is a number that in essence summarizes the difference. µ m - µ f = 5.6 µ m - µ f Z = σ 2 m /N m + σ 2 f /N f Testing Procedure Compute a test statistic. This is a number that in essence summarizes the difference. The possible values of this statistic come from a known probability distribution. According to this distribution, look up the probability of seeing a value meeting or exceeding the test statistic. This is the p value. Lookup probability of test statistic Normal Distribution µ= 0, σ = 1 Z ~ N(0, 1) Z =.2 95% of Probability Mass p > Z > p <

19 Statistical Significance The threshold at which we consider it safe (or reasonable?) to reject the null hypothesis. If p < 0.05, we typically say that the observed effect or difference is statistically significant. This means that there is a less than 5% chance that the observed data is due to chance. Note that the choice of 0.05 is a somewhat arbitrary threshold (chosen by R. A. Fisher) Common Statistical Methods Question Data Type Parametric Non-Parametric Assumes a particular distribution for the data -- usually normal, a.k.a. Gaussian. Does not assume a distribution. Typically works on rank orders. Common Statistical Methods Question Data Type Parametric Non-Parametric Do data distributions 2 uni. dists t-test Mann-Whitney U have different centers? > 2 uni. dists ANOVA Kruskal-Wallis (aka location tests) > 2 multi. dists MANOVA Median Test Are observed counts Counts in χ 2 (chi-squared) significantly different? categories Are two vars related? 2 variables Pearson coeff. Rank correl. Do 1 (or more) variables Continuous Linear regression predict another? Binary Logistic regression Visualization and Statistics Reinforce Each Other 19

20 [The Elements of Graphing Data. Cleveland 94] [The Elements of Graphing Data. Cleveland 94] [The Elements of Graphing Data. Cleveland 94] [The Elements of Graphing Data. Cleveland 94] 20

21 Transforming data How well does curve fit data? Plot the Residuals Plot vertical distance from best fit curve Residual graph shows accuracy of fit [Cleveland 85] [Cleveland 85] Multiple Plotting Options Summary Plot model in data space Plot data in model space Exploratory analysis may combine graphical methods, data transformations, and statistics. Use questions to uncover more questions. Formal methods may be used to confirm, sometimes on held-out or new data. Visualization can further aid assessment of fitted statistical models. [Cleveland 85] 21