Specifying of Requirements for Spatio-Temporal Data in Map by Eye-Tracking and Space-Time-Cube Stanislav Popelka, Vít Voženílek Palacký University, Olomouc, Czech Republic ABSTRACT One of the most objective methods of map use evaluation (in terms of reading, analysis and interpretation. is an analysis of eye movements of map reader, known as eye-tracking method. GazePlots and HeatMaps as the most commonly used visualization methods of eye-tracking data cannot effectively express the change of time. The authors introduce a Space-Time-Cube for spatio-temporal visualization. It displays the map at the base of the cube (axes X and Y) while Z axis is used to represent time. Spatial and temporal components of a map are shown together, and relationship between space and time can be revealed. During the authors research, the user interaction over the map legend of agriculture of the Czech Republic school maps was tested. Space-Time-Cube displayed both components (spatial and temporal) together and allowed easy visual analysis of four respondents (map readers ) work with a map. Using Space- Time-Cube for visual analysis provides satisfactory results, although this form of visualization is not widespread and for someone it may seem complex and confusing. Keywords: Space-Time-Cube, Spatio-Temporal data, Eye-tracking, Eye-movement, Visualization 1. INTRODUCTION Any map can be evaluated from many perspectives. One of the most objective methods of map use evaluation (in terms of reading, analysis and interpretation according to Kimerling et al. [7] is an analysis of eye movements of map reader, known as eye-tracking method. Digital records of eye movements during reading a map can be analyzed as a common data set and can be visualized as a series of spatio-temporal data. The most commonly used visualization methods in the field of eye-tracking are GazePlots and HeatMaps. These methods cannot effectively show the change of point of regard in time. When using two-dimensional visualization methods, visualization of time information is limited and overlapping of newer and older records may occur. It may result in the impossibility of interpreting the phenomenon development for the entire period. The cause of this problem is displaying of the three-dimensional data (X, Y, time) in two-dimensional space (X, Y) [12]. For a more detailed analysis, it is necessary to use methods of spatio-temporal visualization. One of these methods is Space-Time-Cube, which displays the map at the base of the cube (axes X and Y) while Z axis is used to represent time. Spatial and temporal components of a map are shown together, and relationship between space and time can be revealed. By traditional methods of eye-tracking data visualization (GazePlot and HeatMap), these relationships are being remained hidden. 2. SPACE-TIME-CUBE The most important element in the Hägerstrand s spatio-temporal model [3] is Space Time Cube. The concept of Space-Time-Cube is explained by Figure 1. Left part of the figure 1 shows the traditional map containing three colour lines. These lines express the movements of three subjects in Europe in a period of time without time expressions. On the right of the picture, the same situation is displayed using Space-Time-Cube visualization, however with time. The movement of the particular subjects along their routes is shown using time lines in the same colours as the map on the left side. The trajectory of time line clearly indicates the change of the position of the subject on the route. Timelines with direct and perpendicular course to the cube basis express no movement, i.e. no change in position. Steep but not perpendicular course of timeline reflects a slow movement (position change) of a subject over time. The slower (close to horizontal) course of the timeline, the faster change of position of the object/phenomenon.
Figure 1 shows that the red subject has moved quickly between two stations. The green object had moved only twice (with lower speed than the red), and the slowest was the blue subject when moving around Europe. Points and surfaces can be also displayed in the Space-Time-Cube. Fig. 1: The concept of Space-Time-Cube based on time expression using the Z axis. Both images show the same situation. Some contemporary software packages automatically allow creating a Space-Time-Cube from the database. One of them is CommonGIS, developed at the Fraunhofer Institute IAIS. User selections of the elements in the database are immediately reflected in the Space-Time-Cube and vice versa. Current computer technologies allow interactive rotating of the cube and choosing the best perspective for visual data analysis. In CommonGIS, several views of the cube can be displayed in parallel. The main advantage of a Space-Time-Cube is simultaneous displaying of space and time in a uniform environment. 3. APPLICATIONS OF SPACE-TIME-CUBE By Space-Time-Cube, any spatio-temporal data can be displayed. Those can be, for example, data recorded by GPS devices, statistical data with location and time component or data acquired with the eye-tracking technology. Space- Time-Cube visualization can be applied in a variety of different areas. Kraak and Madzudzo [8] applied visualization of point data to the study of Black Death. Sites where plague has been reported were identified as stations. Each station contained an attribute with a time of the beginning and the end of the epidemic and the number of casualties. Displaying of the stations in Space-Time-Cube environment enabled the authors to better understand the development of the epidemic in Europe. It began in the south and gradually expanded to the north. At the bottom of the cube, stations from southern Europe occur almost entirely, whereas upper part of the cube contains mostly stations from the north. More frequent use of Space-Time-Cube is for displaying data using linear Space-Time-Path, especially routes recorded by GPS receivers. Andrienko and Andrienko [1] used Space-Time-Cube, clustering and similarity analysis of individual routes for visualizing and analyzing of the GPS records of 365 routes of one vehicle during the year. Their aim was to determine the distribution of routes within individual days and weekdays throughout the year. Space-Time-Cube visualization of eye-tracking data has been discussed only by [13, 2] and marginally by [9]. The research at Palacký University in Olomouc deals with differences in the perception of 2D and 3D maps [10] or optimization of maps [11]. Maps are evaluated on the basis of the correct composition, map contents or appropriately selected colours. 4. EYE-TRACKING Eye-tracking technology is based on the principles of tracking movements of the human eye while perceiving the visual scene. The measurement device used for measuring eye movements is commonly known as eye-tracker. Eyetracking is one of the methods of usability studies and is considered as an objective because it is not influenced by the
opinion of respondents as the other methods (e.g. questionnaire). Usability can reveal qualities of the product as well as lack of its functionality, which usually arises during the design phase of a product [5]. The modern eye-trackers use contact less (noninvasive) measurements in the visible parts of the eye (pupil, iris and sclera boundary) or corneal reflection of direct beam of infrared light. The reflected light is recorded by camera or other optical sensor. From analysis of the changes of corneal reflection, the point of regard is calculated. The human eye performs several types of movement. The most important are fixations and saccades. During a fixation, eyes are relatively steadily looking at one spot in the visual scene. Irwin [6] states that the average fixation duration is between 150 and 600 ms. The transition between the two fixations is known as saccade. This movement is extremely fast. Saccade usually takes from 30 to 80 ms and we are blind during the most of saccade [4]. Qualitative information about eye movements describes the way in which the user explores the stimulus. It can reveal areas of greatest interest, disruptive elements or search tactics during answering the question. Qualitative information can be obtained, for example, by visual analysis of GazePlots. Quantitative information can be derived from eye-tracking data through metrics of fixation and saccades. They are, for example, the fixation length, saccade amplitude, fixation/saccade ratio or dwell time. This value represents the time spent in predefined areas of interest (AOI). In cartography, AOI can be successfully used for evaluating the composition of maps. These metrics quantify how long each user spent inside composition elements, how many, how long fixation and where fixations were identified or what was the order of visited composition elements. Point of regard of the eye is usually visualized using the GazePlots. GazePlot represents the trajectory of the eye on which the fixations are displayed as circles whose radius corresponds to the length of fixation, and saccades are shown as lines (Fig. 2 right). Visualization of GazePlots for multiple users and for a longer period of time occurs overlaps and possible misinterpretation. Fig. 2: Visualization of raw data from the eye-tracker (left) and their visualization in GazePlot (right). In GazePlot, size of the circles represents the length of fixation. Circles are connected by lines that represent saccades. 5. SPACE-TIME-CUBE AND EYE-TRACKING In the research of map reading during solving the geographic problems at Palacký University in Olomouc, SMI RED 250 eye-tracker developed by SensoMotoric Instrument was used. The device allows data acquisition with frequency of 60 Hz, or 120 Hz. Point of regard of the eye, expressed with Y and Y coordinates is recorded and stored with a regular interval of 8 or 16 milliseconds. During the research, the user interaction over the map legend of agriculture of the Czech Republic school maps was tested. Respondents were asked to quickly find areas where the flax is grown and identify it by clicking the mouse in the map. Total of 16 respondents were participated on the test. Half of the respondents have completed university cartography course, and the rest were non-cartographers. The question was posed to respondents before the stimulus. Respondents have unlimited time to remember it. The time for answering has been set to 45 seconds. Two types of visualization of measured data in a Space-Time-Cube have been tested. The first type was visualization of trajectory made directly from raw. The second type displays the fixations connected with lines (representing saccades). With CommonGIS control feature, the basis of Space-Time-Cube was moved along the Z axis. From the intersection of Space-Time-Paths with the basis plane, the places where the respondent focused were compared. Figure 3 shows that
in the first two seconds of the task both respondents chose almost identical approach of map reading. The first fixations of both respondents are located in the map legend. After that, the respondents sought answers to the question differently and began to search the map by different ways. Fig. 3: Comparison of point of regard trajectories of two respondents viewed from two different angles. From the Space-Time-Cube visualization is evident when respondents proceeded almost identically, and when their trajectories of view were split. Fixation can be visualized as points, but visualization of fixation points connected with lines is highly illustrative. Compared to the previous example, when all vertexes of trajectory of view were connected by lines, there is shown a smaller amount of elements (Fig. 4). Fixation visualization is also not degraded with errors of eye-tracker. These errors are caused by very short loss of data, when eye-tracker writes coordinates [0, 0]. These inaccuracies are filtered out when the algorithm detects fixations. The graphical representation of the length of each fixation can be more illustrative, however the CommonGIS does not include this functionality. Fig. 4: Visualization of the fixations and the links between them. This visualization is clearer than the trajectory of the view. Much smaller amount of data is displayed. 6. CONCLUSIONS Visual analysis of fixation and saccades for several respondents in Space-Time-Cube was processed during the case study. It was found that: regardless cartographic education of the respondents, their first view was directed in the map legend in almost all cases. respondents used a legend several times during the search for answers to the question. one of the questions was constructed so that the respondent had to find the object in the map, but the object was lacking in the legend. Most users spent most of time searching for the object in the map legend.
Most of these conclusions are possible to derive from traditional visualization methods of eye-tracking data. However, only the implementation of Space-Time-Cube can simultaneously show the spatial and temporal component of the data to reveal the sequence of events and the relationship between trajectories of view of individual users. An alternative to traditional methods of data visualization is Space-Time Cube, which displays the map at the base of the cube (axes X and Y) and Z axis is used to represent time. Thanks to spatio-temporal nature of data generated with eyetracking technology, visual analysis of Space-Time-Cube can be used successfully with advantage. The relationship between space and time has been analyzed with the use of Space-Time-Cube. In the case study map reading during answering to the geographic question was testing. Space-Time-Cube displayed both components (spatial and temporal) together and allowed easy visual analysis of four respondents (map readers ) work with a map. Using Space-Time-Cube for visual analysis provides satisfactory results, although this form of visualization is not widespread and for someone it may seem complex and confusing. Although it is possible to statistically evaluate a large number of eye-tracking metrics, visual analysis is a necessary step in the evaluation and optimization of maps using eye-tracking. Visual analysis reveals interesting or problematic areas and issues which require deeper analysis using statistical methods, but it also serves as a stand-alone tool for cognitive cartography. The results of almost all the tests can be visualized in the Space-Time-Cube in addition to traditional visualization techniques. The significant benefit for visual analysis was gained from using of this method. It has considerable potential to reveal spatio-temporal patterns of geographic phenomena that will remain hidden using traditional visualization methods. REFERENCES [1] G. Andrienko and N. Andrienko, Dynamic time transformations for visualizing multiple trajectories in interactive Space-Time-Cube, ICC 2011, (2011) [2] N. Andrienko, G. Andrienko and P. Gatalsky, Visual data exploration using Space-Time-Cube, ICC 2003, 1981-1983, (2003) [3] T. Hägerstrand, What about people in regional science? Papers in regional science, Vol. 26/1, 6-21, (1970) [4] K. Holmqvist, M. Nyström, R. Andersson, R. Dewhurst, J. Halszka and J. Van De Weijer, Eye tracking: A comprehensive guide to methods and measures, Oxford University Press, (2011) [5] M. Hub, O. Víšek and P. Sedlák, Heuristic Evaluation of Geoweb: Case Study. Europan Computing Conference. Proceedings of the European Computing Conference (ECC 11), 142-146, (2011) [6] D. E. Irwin, Visual Memory Within and Across Fixations. In K. Rayner (Ed.), Eye movements and visual cognition: Scene perception and reading. Springer-Verlag, New York, 146 165, (1992) [7] A. J. Kimerling, A. R. Buckley, P. C. Muehrcke and J. O. Muehrcke, Map Use: Reading and Analysis, ESRI Press, Redlands, California, (2009) [8] M.-J. Kraak and P. Madzudzo, Space time visualization for epidemiological research. Proceedings 23rd International Cartographic Conference, 302, (2007) [9] A. S. Nossum and T. Opach, Innovative analysis methods for eye-tracking data from dynamic, interactive and multi-component maps and interfaces, ICC2011, (2011) [10] S. Popelka, A. Brychtová and J. Brus, Evaluation of user preferences during reading of 2D and 3D cartographic visualizations.conference Proceedings SWAET 2012, The Scandinavian Workshop on Applied Eye Tracking Karolinska Instituet, Stockholm, 56, (2012) [11] S. Popelka, A. Brychtová and J. Brus, Advanced Map Optimalization Based on Eye-tracking. Ed.: C. Bateira, Cartography InTech, Rieka, Croatia, (2012) [12] V. Voženílek, Time and Space in Network Data Structures for Hydrological Modelling. In: M. Craglia, H. Onsrud: Geographic Information Research - Trans-Atlantic Perspectives. London, Taylor & Francis, 189-202, (1999) [13] L. Xia, A. Çöltekin and M.-J. Kraak, Visual Exploration of Eye Movement Data Using the Space-Time-Cube, GIScience, 295-309, (2010)