Visualizing Independence Using Extended Association and Mosaic Plots. Achim Zeileis David Meyer Kurt Hornik

Transcription

1 Visualizing Independence Using Extended Association and Mosaic Plots Achim Zeileis David Meyer Kurt Hornik

2 Overview The independence problem in 2-way contingency tables Standard approach: χ 2 test Alternative approach: max test Visualizing the independence problem Association plots Mosaic plots Extensions Visualization & significance testing HCL instead of HSV colors Implementation in grid Multi-way tables The vcd package

3 The independence problem Standard approach: Analyze the relationship between two categorical variables based on the associated 2-way contingency table. Measure the discrepancy between observed frequencies { n ij } and expected frequencies under independence {ˆn ij } by the Pearson residuals: r ij = n ij ˆn ij ˆnij. Use the Pearson X 2 statistic for testing: X 2 = ij r 2 ij, which has an asymptotic χ 2 distribution.

4 The independence problem Alternative approach(es): There are many conceivable functionals λ( ) which lead to reasonable test statistics λ ({ r ij }). In particular: M = max ij r ij. Then, every residual exceeding the critical value c α violates the null hypothesis at level α. Instead of relying on unconditional limiting distributions, perform a permutation test, either by simulating or computing the conditional permutation distribution of λ ({ r ij }).

5 The independence problem Relationship between hair color and eye color among 328 female students: Eye color Hair color Brown Blue Hazel Green Total Black Brown Red Blond Total X 2 = p = 0 M = 6.76 p = 0

6 The independence problem Home and away goals in the Bundesliga in 1995: Away goals Home goals X 2 = p = M = 2.87 p = 0.355

7 The independence problem Treatment for rheumatoid arthritis: Treatment Improvement Placebo Treated Total None Some Marked Total X 2 = p = M = 1.98 p = 0.001

8 Visualization Association plot: display for the Pearson residuals { r ij } and the raw residuals { n ij ˆn ij } in an rectangular array. Mosaic plot: display in which the sizes of the mosaic tiles is proportional to the observed frequencies { } n ij.

9 Visualization Eye Brown Blue Hazel Green Hair Blond Red Brown Black

10 Visualization Eye Brown Blue Hazel Green Blond Red Hair Brown Black

11 HSV colors Colors are commonly used to enhance these plots. In particular, Friendly (1994) suggested shadings for mosaic displays.

12 HSV colors Colors are commonly used to enhance these plots. In particular, Friendly (1994) suggested shadings for mosaic displays. In R these are implemented based on HSV colors. The HSV color space is one of the most common implementations of color in many computer packages. Hue, saturation and value range in [0, 1].

13 HSV colors The hue is typically used to code the sign of the residuals. hue saturation = 1 value = 1

14 HSV colors The hue is typically used to code the sign of the residuals. hue saturation = 1 value = 1 r ij < 0 r ij > 0

15 HSV colors Friendly s extended mosaic displays use the saturation to code the absolute size of the residuals. saturation h = 2/3 h = 0 value = 1

16 HSV colors Friendly s extended mosaic displays use the saturation to code the absolute size of the residuals. saturation h = 2/3 h = 0 value = 1 r ij < 2 2 < r ij < 4 r ij > 4

17 HSV colors Value is currently not used for coding, always set to 1. value h = 2/3 h = 0 saturation = 1

18 HSV colors Value is currently not used for coding, always set to 1. value h = 2/3 h = 0 saturation = 1

19 HSV colors Eye Brown Blue Hazel Green Hair Blond Red Brown Black Pearson residuals:

20 HSV colors Eye Brown Blue Hazel Green Black Pearson residuals: 6 4 Hair Brown Blond Red

21 Visualization & testing HomeGoals Pearson residuals: 0 2 AwayGoals

22 Visualization & testing Intuition: colored cells convey the impression that there is significant dependence.

23 Visualization & testing Intuition: colored cells convey the impression that there is significant dependence. Currently this is not true. But it can be achieved by using the 90% and 99% critical values for the max statistic M instead of 2 and 4. Advantage: color significance highlights the cells which cause the dependence (if any). Disadvantage: does not work for the χ 2 test (or any other functional λ( )).

24 Visualization & testing Eye Brown Blue Hazel Green Black Pearson residuals: 6 4 Hair Brown Blond Red 4 p value: < 2.22e 16

25 Visualization & testing HomeGoals Pearson residuals: 0 2 AwayGoals

26 Visualization & testing Use value to code the result of a significance test for independence. value h = 2/3 h = 0 saturation = 1

27 Visualization & testing Use value to code the result of a significance test for independence. value h = 2/3 h = 0 saturation = 1 non significant significant

28 Visualization & testing Eye Brown Blue Hazel Green Black Pearson residuals: 6 4 Hair Brown Blond Red 4 p value: < 2.22e 16

29 Visualization & testing HomeGoals Pearson residuals: 0 2 AwayGoals p value:

30 HCL colors Disadvantages of HSV colors: device dependent, not copierproof, flashy colors good for drawing attention to a plot, but hard to look at.

31 HCL colors Disadvantages of HSV colors: device dependent, not copierproof, flashy colors good for drawing attention to a plot, but hard to look at. Alternative: use HCL colors instead (see Ihaka, 2003). HCL colors are defined by hue (in [0, 360]), chroma and luminance (in [0, 100]). HCL space essentially looks like a double cone.

32 HCL colors

33 HCL colors

34 HCL colors

35 HCL colors

36 HCL colors

37 HCL colors

38 HCL colors

39 HCL colors

40 HCL colors

41 HCL colors

42 HCL colors

43 HCL colors

44 HCL colors

45 HCL colors hue = 0 hue = 260 luminance chroma

46 HCL colors hue = 0 hue = 260 chroma luminance

47 HCL colors hue = 0 hue = 260 chroma luminance

48 HCL colors hue = 0 hue = 260 chroma luminance

49 HCL colors hue = 0 hue = 260 chroma luminance

50 HCL colors hue = 0 hue = 260 chroma luminance significant

51 HCL colors hue = 0 hue = 260 chroma luminance significant non significant

52 HCL colors Eye Brown Blue Hazel Green Black Pearson residuals: 6 4 Hair Brown Blond Red 4 p value: < 2.22e 16

53 HCL colors Eye Brown Blue Hazel Green Black Pearson residuals: 6 4 Hair Brown Blond Red 4 p value: < 2.22e 16

54 HCL colors HomeGoals Pearson residuals: 0 2 AwayGoals p value:

55 HCL colors HomeGoals Pearson residuals: 0 2 AwayGoals p value:

56 HCL colors Treatment Placebo Treated Pearson residuals: None 1 Improved 0 Some 1 Marked p value:

57 HCL colors Treatment Placebo Treated Pearson residuals: None 1 Improved 0 Some 1 Marked p value: 0.001

58 Implementation in grid The graphics engine grid overcomes the old R concept of plots with a plot region surrounded by a margin. grid is based on generic drawing regions (viewports), allows for plotting to relative coordinates, is also the basis for an implementation of Trellis graphics called lattice. (see Murrell, 2002) Thus, the new implementation of mosaic and association plots makes them easily reusable, e.g., in Trellis-like layouts.

59 Implementation in grid Furthermore, graphics parameters for the rectangles, e.g., fill color, line type, line color, can be specified for each cell individually by the user. Each graphics parameter can be an object of the same dimenionality as the original table. new shadings can easily be implemented.

60 Multi-way tables Dept = A Dept = C Dept = E Admit Reject Admit Reject Admit Reject Gender Female Male Dept = B Admit Reject Female Male Female Male Dept = D Admit Reject Female Male Female Male Dept = F Admit Reject Female Male Admit

61 The vcd package New methods will be available in the package vcd for visualizing categorical data. Currently only in development version. The released version is available from the Comprehensive R Archive Network and it already offers some functionality for fitting & graphing of discrete distributions, plots for independence and agreement, visualization of log-linear models.