Highlighting in Geovisualization. Anthony C. Robinson

Transcription

1 Highlighting in Geovisualization Anthony C. Robinson ABSTRACT: Coordinated view geovisualizations allow users to interactively pick and attend to data observations across multiple views. This is frequently supported by the transient application of a visual effect to an observation during a mouse selection or rollover. This technique, known as highlighting, is typically implemented using a dedicated bright and saturated color to outline observations. In this paper we present a range of possibilities for alternative approaches to color highlighting, beginning with examples from the range of available visual variables and moving beyond those options to other, non-visual variable methods such as the use of lines to connect highlighted observations. We also describe design criteria for highlighting methods that can be used to predict and test the suitability of different approaches, and apply those criteria to our set of proposed methods to identify potential good candidates for implementation in future systems. Next, we present a set of highlighting types that define basic ways in which highlighting methods can be combined, modified, and driven directly by data values. We conclude by outlining several broad research challenges for future work on the development and evaluation of highlighting methods in geographic visualization. KEYWORDS: Highlighting methods, geographic visualization Introduction Multiple, coordinated-view geographic visualizations allow users to explore and analyze spatial data represented using a wide range of forms. One of the fundamental interactive behaviors of such systems is the use of visual methods for interactively highlighting observations across multiple views. In most current geographic visualization systems highlighting is implemented using a specific bright and highly- saturated color to outline or fill observations (Figure 1). As data and the number of views we use to explore those data become increasingly complex (Thomas and Cook 2005), it is time to explore new highlighting methods to compare alternative approaches to the use of color and ensure that visually-enabled analysis is efficient and effective. To date there has been little work focused specifically on extending available highlighting methods beyond the simple use of a bright color. This research is intended to explore and Anthony C. Robinson, GeoVISTA Center, John A. Dutton e-education Institute, Department of Geography, 302 Walker Building, The Pennsylvania State University, University Park, PA arobinson@psu.edu DOI: characterize the realm of possibilities and to predict (to some extent) their theoretical usefulness in order to suggest future implementation efforts and comparative evaluation strategies for highlighting approaches. The key contributions of this research are a set of proposed design criteria for functional and analytically effective highlighting methods, an extended set of highlighting techniques (building on our earlier work (Robinson 2009)) made possible by exploring all available visual variables as well as other methods, a set of interaction types that define basic ways in which highlighting methods can be combined and modified, and a theoretical framework that matches highlighting methods to their predicted utility/usability based on the aforementioned design criteria. Ultimately, this research provides geovisualization developers with a theoretical basis for choosing and applying visual highlighting methods, and it provides a leverage point for future work focused on identifying and evaluating visual methods to support geographic exploration and analysis. Cartography and Geographic Information Science, Vol. 38, No. 4, 2011, pp

2 Figure 1. Most current systems use color to visually link items across multiple, coordinated views. Background Defining highlighting is a challenge because it is closely linked to the concept of brushing, which is the mechanism by which observations are dynamically queried and selected in coordinated view visualizations (Becker and Cleveland 1987; Roberts and Wright 2006). Becker and Cleveland s (1987) early work on defining brushing calls this effect transient paint. In the geographic visualization literature this technique has been called indication, where it is described as a transient selection, typically as a result of a mouseover (MacEachren, Hardisty et al. 2003). Recent work in information visualization suggests the term highlighting (Seo and Shneiderman 2004; Ware and Bobrow 2005; Roberts and Wright 2006), and here we have chosen to use this term to describe the transient visual effect that is applied on observations across views as a result of brushing. In most current geographic and information visualization systems, highlighting is implemented through the use of static color outlines or fills on observations in multiple views. Examples of systems that use color highlighting include the GAV Toolkit (Jern, Johansson et al. 2007), GeoViz Toolkit (Hardisty and Robinson 2011), ArcGIS (Esri 2010), Prefuse (Heer, Card et al. 2005), SpotFire (Ahlberg 2006), Jigsaw (Stasko, Gorg et al. 2007), and InfoVis Toolkit (Fekete 2004). Our focus on highlighting methods is closely related to common frameworks for the design of information visualizations. Baldonado s Rule of Self-Evidence (Baldonado, Woodruff et al. 2000) for coordinating multiple views states that designers should take advantage of, perceptual cues to make relationships among views more apparent to the user. Highlighting is also a key enabling feature of the visual informationseeking mantra of Overview First, Zoom and Filter, Details on Demand (Shneiderman 1996), as an interactive visual cue is often an essential part of identifying which observations to choose from an overview to zoom and filter. One of the few examples of research that focuses specifically on highlighting is work by Ware and Bobrow (Ware and Bobrow 2004; Ware and Bobrow 2005) on the use of motion as an alternative to static highlighting. This work compared different types of movement to static color highlighting in terms of user performance with node-link graphs. Their results suggest that motion and static color highlighting methods are equally effective, and that when they are used in combination they can also be quite effective. These findings beg the question as to whether or not other possible static highlighting methods would perform similarly. Related work by Roberts and Wright (2006) discusses opportunities for extending the concept of brushing beyond views and into interfaces, to support what they call ubiquitous brushing. As part of their proposed framework, Roberts and Wright call for research on highlighting since most current systems use color despite the fact that other alternative visual approaches are possible. At the broadest level, our focus on highlighting is really to identify alternative approaches for capturing visual attention in geographic visualization systems. Wolfe and Horowitz have recently identified and ranked a range of possible candidates for attracting visual attention, characterizing twenty-eight methods that include static and dynamic visual effects according to extant evidence to support their use for capturing visual attention (Wolfe and Horowitz 2004). Color, shape, orientation, and motion, are grouped together in the undoubted attribute category, indicating that there is broad consensus that these effects have attributes that guide attention. Our initial work on exploring alternative highlighting approaches proposed six new methods that move beyond the simple use of color Vol. 38, No

3 (Robinson 2009). Here we extend that initial work by defining design criteria for evaluating the utility of highlighting methods, an expanded set of fifteen possible highlighting methods, a set of interaction types that can be used to combine and modify highlighting methods, and by providing a theory- based evaluation of potential methods that identifies good candidates for alternatives to color highlighting. Design Criteria for Highlighting Methods Here we propose two basic categories of design criteria for highlighting methods that refer to basic functional and analytical properties (Figure 2). Functional factors are criteria to ensure that highlighting methods can be potentially utilized in geographic visualizations, while analytical factors are criteria to ensure that highlighting methods are useful for data interpretation and analysis. Functional Design Criteria In terms of basic functionality, highlighting methods must meet three basic criteria. First, they must be visually salient, so that highlighted objects are clearly visible by end-users. Second, a successful highlighting method should be usable across a broad range of common visualization forms (e.g. maps, scatterplots, tag clouds, and parallel coordinate plots). Finally, highlighting methods should be implementable without serious performance issues (ensuring refresh rates of at least 10 frames per second (Thomas and Cook 2005)) to support fluid and dynamic user interaction. Analytical Design Criteria Beyond basic functionality, highlighting methods should support the visually-led analytical process by conforming to three essential criteria. First, highlighted observations must not change shape or size as a result of highlighting, as this has the potential to fundamentally change how observations are interpreted. Second, visuallyrepresented attributes of observations should be preserved. For example, the color associated with a line in a parallel coordinate plot may correspond to its membership in a category, and obscuring this color with a highlighting method would impact its use in analysis. Finally, the context around a highlighted object (neighboring observations) should be preserved to support pattern analysis. Design Criteria For Highlighting Methods Functional Design Criteria Visually Salient Usable Across A Broad Range of Representational Forms Possible to Implement Without Performance Penalties Analytical Design Criteria Preserve Observation Size/Shape Preserve Observation Symbolization Preserve Nearby Observations For Context Figure 2. Design criteria for highlighting methods. Highlighting Methods Based on Visual Variables To propose a comprehensive set of possible new approaches to highlighting, we start with the full set of available visual variables. A basic set of visual variables for graphically representing data were first proposed by Bertin ( ) and this set was later expanded by others to include additional methods (Morrison 1974; Mackinlay 1986; MacEachren 1995) and to expand its focus to dynamic visual variables (Carpendale 2003). While the intended use of visual variables is to design data representations, our goal is to take a broad approach to identifying possible candidates for highlighting methods and to do so we begin by exploring how the twelve visual variables compiled by MacEachren (MacEachren 1995) may be used as highlighting methods. Figure 3 shows graphical mockups of common visual variables applied as highlighting methods to point, line, polygon, and text observations in an attempt to cover the range of broadly used representation forms in geographic visualizations. While highlighting is an interactive transient effect (and therefore cannot be easily judged using static images) this figure shows how some highlighting methods may apply more readily to all four observation types than others. For 376 Cartography and Geographic Information Science

4 Figure 3. Highlighting methods based on common visual variables. Vol. 38, No

5 example, color, focus (Kosara, Miksch et al. 2002), and transparency can be readily applied to all observation types, while location, size, and arrangement seem to work only with a subset of observation types. In Section 6 we delve more deeply into the task of identifying which methods are potentially good candidates for implementation based on our proposed design criteria. Other Highlighting Methods Aside from the commonly-accepted set of visual variables, it is also possible to create highlighting methods that make use of other visual forms. Here we propose three such methods: leader lines, style reduction, and contouring. We do not pretend to have exhausted all of the possible options here rather these are a starting point for what are doubtless many other ways of highlighting aside from using common visual variables. The leader line method is inspired by a long tradition in cartographic design of using lines to point from labels to the objects they represent (Slocum, McMaster et al. 2005). We propose leader line highlighting to draw lines outward from the mouse cursor location to link observations in multiple views (Figure 4). This method is already implemented in the Jigsaw system to connect related entities that have been identified from text reports (Stasko, Gorg et al. 2007). on topographic maps to represent terrain relief (Slocum, McMaster et al. 2005). Some systems like SpotFire (Ahlberg 2006), draw outlines at a distance from highlighted observations to create a contouring effect. Figure 5. An example of contour line highlighting. Style reduction highlighting is based on the assumption that many visual representations include a variety of display marks used to label and outline objects. Style reduction highlighting works by eliminating all but the most basic representational form on non-highlighted objects when a selection is made (Figure 6). This method will only work with representations that can be pared back to reduce their visual complexity. Figure 6. Style reduction highlighting (1 prior to mouseover, 2- mouseover on PA). Figure 4. An example of leader lines used for highlighting. Contouring can also be used to create the pseudo 3-dimensional effect of raising a highlighted object from its surrounding neighbors (Figure 5). This method also connects to a cartographic technique; in this case the use of contour lines Identifying Potentially Effective Highlighting Methods The results of previous research efforts to characterize the suitability of visual variables for representing different data types can be used alongside the design criteria we define in Section 3 to identify which highlighting approaches may be good candidates for future implementation. Specifically, here we make use of prior work that proposes the degree to which visual variables support Bertin s principles of selectivity and 378 Cartography and Geographic Information Science

6 associativity (Bertin ) (later called visual isolation and visual levels by MacEachren (1995)). A visual variable is selective if it allows a particular observation to be visually isolated. A visual variable is associative if it supports grouping on a specific visual characteristic (like the shape of scatterplot marks, for example). These two criteria can be used to characterize the full range of highlighting approaches in terms of their visual salience when used in geographic visualizations. To help further rank highlighting methods by their theoretical effectiveness, we can also include ratings derived from Wolfe and Horowitz who have characterized the ability of a wide range of methods for capturing visual attention (Wolfe and Horowitz 2004). This work assigned twenty-eight methods (including static as well as dynamic visual effects) to five categories according to the likelihood that they have attributes that can guide attention. Figure 7 shows a graphical summary of highlighting methods and their projected effectiveness based on our judgment on whether or not they meet the design criteria we proposed in Section 3 as well as how well they are expected to be visually salient based on the results of the aforementioned earlier work by MacEachren (1995) and Wolfe and Horowitz (2004). Each design criteria category is colored to indicate the degree to which a particular highlighting method meets that design criteria (good, marginal, poor, or n/a). In some cases, previous work to quantify the visual salience of a particular approach (leader lines, for example) is not available and therefore is categorized accordingly. Finally, the implementation performance criteria is omitted from this summary as meeting this criteria will always depend on the specific software application in question and its performance capabilities. This graphical summary of the potential effectiveness of our proposed highlighting methods reveals that none of the methods fully satisfy all five criteria. However, five methods (hue, value, saturation, focus, and leader line) meet four of the criteria and are therefore good candidates for future implementation and testing. Several other methods meet three criteria including the ability to work across all representational forms (transparency, resolution, contouring), which we feel is the second most important criteria after visual salience. We predict that shape, arrangement, texture, orientation, size, and location are the poorest candidates for future implementation in geographic visualization systems. Each of these methods does a poor job on meeting at least one of the design criteria. While our intention is to identify potential good candidates for alternative highlighting methods, we recognize that some methods may work well for certain data representation forms better than others, and not all toolkits feature all four of the key representation types we have explored here. For example, using orientation, texture, arrangement, and shape is quite difficult to conceive in applications that feature textual and linear representations, but these methods are plausible for polygon and point representations. Another important limitation is our knowledge of the extent to which methods like leader lines, style reduction, and contouring are visually salient. While one can imagine these methods capturing at least some visual attention, this remains to be investigated. Furthermore, with any visual highlighting technique there may be a large number of possible implementation choices that can impact their utility and usability. For example, the size of leader lines may be adjusted, the amount of blur used for focus highlighting can be changed, and the degree of color saturation can be modified. Approaches to Highlighting Here we describe methods by which highlighting methods can be applied, combined, and connected to data sources. The simplest form of highlighting applies a single highlighting method to all views (e.g. most current systems that use color highlighting). However, it is also possible to apply different highlighting methods in different views, perhaps depending on which methods are most salient with particular types of representations. Additionally, it is possible to envision coupling multiple highlighting methods together (e.g. color highlighting and leader lines) to create a compound technique (Figure 8). It is also possible to envision data-driven highlighting methods that tailor the transient Vol. 38, No

7 Figure 7. Potential effectiveness of proposed highlighting methods. 380 Cartography and Geographic Information Science

8 Figure 8. Combining color, focus, and transparency highlighting. visual effect according to one or more data values that accompanies an observation. For example, one analytical advantage provided by exploring data in multiple, coordinated views is that it becomes possible to identify and characterize the context in which an observation appears (e.g. neighboring observations in a scatterplot that make up a cluster, or multiple counties on a map that make up a region). One potential way to connect highlighting more directly to the task of exploring context is to tie highlighting to data classes (Figure 9). Figure 9. A thick leader line connects observations in two views, while thinner lines radiate out to categorical neighbors. Evaluation, Implementation, and Research Challenges Our work suggests several broad research challenges for the development and evaluation of highlighting methods in geographic visualization. A priority objective is to identify whether or not there are differences in user performance among the range of highlighting methods. Evidence from prior work suggests that motion and static highlighting methods together may offer advantages (Ware and Bobrow 2005), but little effort has focused so far on comparing static highlighting methods to each other. As a starting point, we have recently begun work on an eye-tracking evaluation method for comparing static highlighting methods in terms of how fast a user attends to highlighted objects as well as answer correctness for basic analytical questions that require the discovery and comparison of highlighted observations (Griffin and Robinson 2010). This work has the initial goal of comparing color highlighting to leader lines in two view combinations; choropleth map linked to a scatterplot, and choropleth map linked to a parallel coordinate plot (PCP). Our evaluation method makes use of static images that show these view combinations and highlight observations in views using color outlines or leader lines. Study participants view these images and are asked to complete a simple task to verbally state the name of the highlighted observation in the map (labeled with a simple 2-letter title) and estimate its value for one of the variables in the scatterplot or PCP. During the study, an infrared eye-tracker that sits underneath the computer monitor tracks eye-movements for each participant and the screen is recorded so that an overlay of eye-fixations can be evaluated later. A webcam records audio and video of each participant as well. The balanced, withinsubjects study requires participants to complete 32 such tasks (8 tasks for each combination of views and highlighting methods). To analyze the eye-tracking data, we calculate the time it takes for users to fixate on the highlighted map observation, to fixate on the highlighted target in the scatterplot/ PCP, and to answer the question with a verbal response. These data allow us to calculate how fast participants attend to the highlighted observations for each method, and to compare each method in terms of task accuracy. Data gathered from this study are still being analyzed, but early results indicate that user performance differs by highlighting method and also depends on the view combinations in question, but that in general there are few significant differences between performance and accuracy for color and leader line highlighting methods. This suggests that the latter may well be a solid substitute for traditional color highlighting in coordinated visualization systems. Beyond basic task performance and accuracy, other challenges associated with comparing highlighting methods include measuring performance in different sizes/types of views and Vol. 38, No

9 the need to compare different visual parameters for highlighting (e.g. the amount of blur used in depth-of-field highlighting or the width of leader lines). Identifying effective default settings for each highlighting method will be an important milestone to achieve before many of these techniques can see widespread adoption in realworld visualization systems. Additional evaluation-focused challenges include the need to identify how well highlighting works as screen sizes increase, the number of coordinated views increases, and the density of represented information increases. It is likely that compound highlighting and/or dynamic highlighting methods will be necessary to support effective visually-led exploration and analysis in very complex systems. Determining where methods break down in performance and possible solutions for those problems will be important goals for future research. In terms of implementation it remains to be seen how well some of the proposed methods can be integrated in current systems that already incorporate some form of basic highlighting across multiple views. The leader line method may be problematic for systems that use windows for each view, since lines need to draw across views and will need to cross over those frames. The focus method may impose performance penalties on already taxed systems since it will require rapid rasterization in each view from one mouseover event to the next. Early implementation of some of the proposed highlighting techniques in this paper has been accomplished in the GeoViz Toolkit (Hardisty and Robinson 2011), where leader lines, focus blurring, and transparency have been combined to help users quickly visually evaluate patterns found in complex coordinated views (Figure 10). This example implements a compound highlighting method which to our knowledge has not been attempted in other geovisualization Figure 10. Compound highlighting using leader lines, focus, and transparency has been implemented in a recent modification of the GeoViz Toolkit (Hardisty and Robinson 2011). In this example, results from the 2008 U.S. Presidential election have been interactively filtered to focus on an area of the northeast. Leader lines connect the highlighted county in Michigan to corresponding highlighted observations in the other views. Focus and transparency are used to visually separate the target region in the map, PCP, and scatterplot views. 382 Cartography and Geographic Information Science

10 systems. In the future we are likely to use the GeoViz Toolkit as an interactive platform to support new eye-tracking studies to evaluate combinations of highlighting methods and their suitability for coordinated visually-enabled analysis. Finally, it is possible that a visual method for highlighting observations may not always be the best possible solution. Sonic and haptic methods have yet to be fully explored in this context (Keehner and Lowe 2010). It is also possible that scale-related issues with respect to screen size, number of views, and the density/complexity of observations may be best managed with visuallyenabled computational methods to automatically cluster and generalize information displays to reduce visual overhead through automated means. Conclusion In this paper we have presented a set of possible alternatives to the color highlighting method used in most current geographic visualization systems. We have also outlined design criteria to use for developing and evaluating new highlighting methods and we have used those criteria to take the first step toward identifying potential good candidates for future implementation in geographic visualization systems. In addition, we presented several ways in which highlighting methods can be applied that move beyond the simple approaches used in most current systems. In addition to highlighting methods that make use of common color components (hue, value, saturation), our work indicates that good likely candidates for implementation in future systems include focus, leader lines, transparency, resolution, and contouring. While our work here is limited to static, visual techniques (and we have no doubt left much work ahead to account for the diverse range of combinations, twists, and possibilities there) we recognize that there is ample potential to explore the possibilities of new dynamic, sonic, and haptic highlighting methods. This work provides a starting point toward future research to implement, design, and evaluate new highlighting methods. The next steps for extending this research to further explore highlighting methods are clear; implement each of these techniques in real systems and further evaluate their performance in controlled and task-oriented studies. References Ahlberg, C Spotfire DecisionSite. Somerville, MA. Baldonado, M.W., A. Woodruff and A. Kuchinsky Guidelines for using multiple views in information visualization. 5th International Working Conference on Advanced Visual Interfaces, Palermo, Italy, ACM Press. Becker, R.A. and W.S. Cleveland Brushing Scatterplots. Technometrics 29(2): Bertin, J. 1967; Semiology of Graphics: Diagrams, Networks, Maps. Madison, WI, University of Wisconsin Press. Carpendale, M.S.T Considering visual variables as a basis for information visualization. Calgary, Canada, University of Calgary. Environmental Systems Research Institutue (ESRI) ArcGIS 10. Redlands, CA, ESRI, Inc. Fekete, J.D The InfoVis Toolkit. IEEE Symposium on Information Visualization, Austin, TX. Griffin, A.L. and A.C. Robinson Comparing color and leader line approaches for highlighting in geovisualization. GIScience, Zurich, Switzerland. Hardisty, F. and A.C. Robinson The GeoViz Toolkit: using component-oriented coordination methods to aid geovisualization application construction. International Journal of Geographical Information Science 25(2): Heer, J., S.K. Card and J.A. Landay Prefuse: a toolkit for interactive information visualization. ACM SIGCHI conference on Human factors in Computing Systems, Portland, OR. Jern, M., S. Johansson, J. Johansson and J. Franzen. (2007). The GAV Toolkit for Multiple Linked Views. Fifth International Conference on Coordinated and Multiple Views in Exploratory Visualization, London, England. Keehner, M. and R.K. Lowe Seeing with the hands and with the eyes: the contributions of haptic cues to anatomical shape recognition in surgery. Association for the Advancement of Vol. 38, No

11 Artificial Intelligence Spring Symposium. Stanford, CA. Kosara, R., S. Miksch and H. Hauser Focus+Context Taken Literally. IEEE Computer Graphics and Applications (CG&A), Special Issue on Information VIsualization 22(1): MacEachren, A.M How Maps Work: Representation, Visualization and Design. New York, Guilford Press. MacEachren, A.M., F. Hardisty, X.P. Dai and L. Pickle Supporting visual analysis of federal geospatial statistics. Communications of the ACM 46(1): Mackinlay, J Automating the design of graphical presentations of relational information. ACM Transactions on Graphics 5(2): Morrison, J.L A theoretical framework for cartographic generalization with the emphasis on the process of symbolization. International Yearbook of Cartography 14: Roberts, J.C. and M.A.E. Wright Towards Ubiquitous Brushing for Information Visualization. IEEE International Conference on Information Visualization, London, England. Robinson, A.C Visual highlighting methods for geovisualization. 24th International Cartographic Conference, Santiago, Chile. Seo, J. and B. Shneiderman A Rankby-Feature Framework for Unsupervised Multidimensional Data Exploration Using Low Dimensional Projections. IEEE Symposium on Information Visualization, Austin, Texas. Shneiderman, B The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations IEEE Conference on Visual Languages, Boulder, CO. Slocum, T.A., R.B. McMaster, F.C Kessler and H.H. Howard Thematic Cartography and Geographic Visualization. Upper Saddle River, NJ, USA, Pearson Prentice Hall. Stasko, J., C. Gorg and Z. Liu Jigsaw: supporting investigative analysis through interactive visualization. IEEE Symposium on Visual Analytics Science and Technology (VAST 2007), Sacramento, CA. Thomas, J.J. and K.A. Cook, (Eds.) Illuminating The Path: The Research and Development Agenda for Visual Analytics. New York, IEEE CS Press. Ware, C. and R. Bobrow Motion to Support Rapid Interactive Queries on Node- Link Diagrams. ACM Transactions on Applied Perception 1(1): Ware, C. and R. Bobrow Supporting visual queries on medium sized node-link diagrams. Information Visualization 4(1): Wolfe, J.M. and T.S. Horowitz What attributes guide the deployment of visual attention and how do they do it? Nature Reviews Neuroscience 5(6): Cartography and Geographic Information Science