Accepted by: Annals of the Association of American Geographers. Spatial Information Theory Meets Spatial Thinking - Is Topology the Rosetta Stone

Transcription

1 Accepted by: Annals of the Association of American Geographers Spatial Information Theory Meets Spatial Thinking - Is Topology the Rosetta Stone of Spatial Cognition? Alexander Klippel

2 Abstract. Topology is the most commonly used spatial construct to bridge the gap between formal spatial information theory and systems on the one side and (human) spatial cognition and thinking on the other. To this end, we find topological calculi in virtually all research areas pertinent to spatial information science such as ontological modeling, geographic information retrieval, image analysis and classification. Manifold experiments have been conducted to assess the cognitive adequacy of topological calculi with varying results. Our contribution here is unique for two reasons: on the one hand, we are addressing, behaviorally, the role of topology in the crucial area of spatio-temporal information; on the other hand, we are evaluating the role of topology across different semantic domains. We report five experiments that were conducted in the framework we developed (Klippel and Li 2009), which combine critical constructs from spatial information theory and cognitive science. Topologically equivalent movement patterns were specified across five domains using paths through a conceptual neighborhood graph. This approach allows us to disentangle the role of topology from the influence of semantic context. The results show that topology plays an important yet not semanticindependent role in characterizing the cognitive conceptualization of geographic events. Keywords: Qualitative spatial reasoning, spatio-temporal information, spatial cognition; conceptual neighborhood graphs.

3 Introduction Spatial information science (SIS) has developed qualitative formalisms to represent and reason with spatial information. A large number of these research efforts explicitly target the gap that exists between the requirements of formal symbol processing systems (e.g., a computer) and those of natural cognitive systems (e.g., humans) for understanding and interacting with information (Hobbs and Moore 1985; Worboys, Duckham, and Kulik 2005; Kurata and Egenhofer 2009). In a society in which computers are ubiquitous and in which ambient intelligence is gaining central importance (Gottfried and Aghajan 2009), the seamless interaction and integration of computational and cognitive systems is a vision, dream, and challenge at the same time. But how do we cognitively ground qualitative formalisms? An important aspect to understand for those seeking to cognitively ground qualitative formalisms is that humans are embedded in their spatial environments, and understanding and formally characterizing how humans process spatial information is essential to improving spatial information theories and systems. An equally important point is that through embodied interactions with spatial environments, humans develop fundamental concepts that enable their comprehension of more abstract concepts such as time. Spatial is special and while SIS and geography are the primary disciplines to acknowledge this, several disciplines such as the cognitive sciences have given space a prominent role in their theoretical frameworks (Cohen 1985; Carlson, Hölscher, and Shipley 2011).

4 The role that SIS plays is twofold: on the one hand, SIS integrates research on how humans understand their spatial environments into its theories; on the other hand, SIS provides theories that help to formally explain what is meant when other disciplines refer to space and spatial relations. This article addresses formal theories centering on topology, the most commonly used approach to establish cognitive adequacy for spatial information theories. It is crucial to note that psychologists, linguists and cognitive scientists acknowledge the importance of topology as probably the single most important spatial concept fundamental to cognition in general, not just spatial cognition (Piaget 1955; Klix 1971; Johnson 1987; Lakoff 1987; Jackendoff and Landau 1992; Mandler 1992; Bowerman 2007). Klix (1971) pointed out that one of the reasons contributing to the centrality of topology is that a topological characterization allows for naming invariants that can be identified in spatial environments both formally and from the perspective of cognitive information processing. But, what exactly is meant when cognitive scientists talk about topology? To unleash the full potential that SIS has in explaining spatial cognitive processes, novel techniques for experimental designs (behavioral/cognitive evaluations) and analysis techniques that allow for relating cognitive and formal perspectives more directly are needed. It is important to acknowledge and harvest the potential of SIS: On the basis of formal theories we can render notions of spatial information (such as topology) precise enough to formulate testable hypotheses. In other words, we can design tailored and precise experiments that are critical to understanding human spatial cognition. In return, theories in SIS will be evaluated and the often-used label, cognitively adequate, becomes more meaningful.

5 The problem, simply put, is that formal qualitative characterizations of a spatial environment are by default claimed to be cognitively adequate, that is, they are capturing spatial aspects important to a cognitive system on the assumption that qualitative = cognitive. Of course, as part of humans common sense knowledge (Hobbs and Moore 1985; Davis 1990; Egenhofer and Mark 1995b, 1995b; Worboys, Duckham, and Kulik 2005), humans think about the world largely qualitatively, not quantitatively. However, the assumption that, from a formal perspective, everything qualitative equals cognitive is not scientifically defensible. Hence, the main problem from the perspective of this article is that for the most part, qualitative formalisms escape the scrutiny of behavioral/cognitive validation. One of the most often found concluding remarks in articles that develop qualitative formalisms is that human participant experiments are needed to validate the proposed formalism. Such follow-up studies almost never happen. There are a few noteworthy exceptions especially in the area of validating topological calculi. First and foremost the extensive work by Mark and Egenhofer (Mark and Egenhofer 1994a, 1994b; Shariff, Egenhofer, and Mark 1998; Mark 1999) that addressed the validity of Egenhofer s 9-intersection model as a framework to model the cognitive understanding of spatial relations. In their research they coined the famous expression: topology matters and metric refines. In summary, their findings assign topology the most prominent role in a high-level, conceptual understanding of spatial relations. The example domain they focused on is a road (statically represented by a line) and a park. Other studies that come to similar conclusions, that is, that topology is crucial for modeling the cognitive understanding of spatial relations were conducted by Knauff and collaborators (1997), Zhan (2002), Riedemann (2005), Xu (2007) and Wang and

6 collaborators (2008). Additionally, we find related research on Allen s relations (Allen 1983) that complementarily addresses the role of qualitative temporal characterizations (Lu, Harter, and Graesser 2009; Matsakis, Wawrzyniak, and Ni 2010). The following aspects are important to observe in the above mentioned experimental validations: First, many of them use abstract geometric figures as stimuli rather than real world scenarios. In case they do use real world scenarios the scope is limited to one particular scenario. Second, with the exception of Lu and collaborators, the stimuli used in the above mentioned experiments are static, even though they may be seen as dynamic phenomena. Using abstract geometric figures ignores findings from cognitive science that there is an interplay between bottom-up and top-down information processing (Neisser 1976; Zacks 2004). In other words, taking domain semantics into account may change the invariants in a spatial environment identified by a cognitive system. The approach to incorporate both top-down and bottom-up characteristics also corresponds to the rapidly developing area of assessing and formalizing contextual effects (e.g., Dey 2001; Cai in press) that we will not further discuss here. ====== Figure 1 ======

7 To further illustrate this aspect, Figure 1 demonstrates this incongruity by employing the Rosetta Stone as an example. The Rosetta Stone is seen as one of the most crucial artifacts in human history for deciphering Egyptian writing. The beauty of the stone is that it bears three translations of the same passage of a decree issued at Memphis, Egypt in 196 BC on behalf of King Ptolemy V. (Wikipedia, retrieved April 21 st 2011): one is the relatively well-known classic Greek, the others (hieroglyphic and Demotic) were able to be deciphered based on their correspondence to the Greek passage. Figure 1 shows the Rosetta Stone overlaid with a topology-based conceptual neighborhood graph (CNG) (Egenhofer and Al-Taha 1992; Freksa 1992). CNGs are graph structures based on qualitative spatial and temporal formalisms. They were first proposed for Allen's (1983) temporal intervals by Freksa (1992) and were quickly adapted to corresponding qualitative spatial calculi (Egenhofer & Al-Taha, 1992; Muller, 1998). Conceptual neighbors are defined as two relations (e.g., disconnected, DC, and externally connected, EC) that can be directly transformed into one another by continuous topological transformations such as shortening, lengthening, or moving (translation). The importance of CNGs come from the fact that virtually all qualitative calculi that specify jointly exhaustive and pairwise disjoint (JEPD) relations have conceptual neighborhood graphs (see Cohn and Renz 2008). In Figure 1, the Rosetta Stone/CNG is surrounded by depictions of trajectories of moving entities from different semantic domains. This example illustrates what has been discussed in the preceding section regarding the invariants in spatial cognition and the importance of topology: each of these trajectories, as different as they may be from the perspective of Euclidean metric, speed, or background knowledge (semantics), is

8 identical from a topological perspective. That is, if we apply a topological characterization (Fernyhough, Cohn, and Hogg 2000; Galton 2000; Muller 2002; Kurata and Egenhofer 2009), all trajectories are topologically equivalent (identical paths through the conceptual neighborhood graph) from a topological perspective they are invariant. While this approach potentially formalizes the semantics of spatial expressions such as the verb cross or the preposition across, the critical question is if the trajectories are indeed all meaningful in the same way (in all semantic domains), and whether they have the same meaningful, topologically identified subparts. To make the analogy to the Rosetta Stone, can topology (as the Rosetta Stone) be used to translate between movement patterns from different semantic domains? That is, are topological relations equally important and are topologically identified invariants universal across semantic domains? If topology were indeed the Rosetta Stone of Spatial Cognition (including the conceptualization of movement patterns), this paper would end here. However, the number of formalisms for topology alone is already diverse and debated within SIS and associated areas of artificial intelligence. For example, the two most often quoted frameworks for characterizing the relationship between two spatially extended entities, the region connection calculus (Randell, Cui, and Cohn 1992) and the 4- and 9- intersection models (4IM / 9IM) (Egenhofer and Franzosa 1991), offer two levels of granularity distinguishing either eight or five topological relations. It is important to note that while the fine levels of granularity match and make the same distinctions, the coarse levels of granularity do not (for details see Discussion Section). In a similar vein, Clementini et al. (1993) deemed the eight topological relations distinguished in 4IM/9IM

9 as cognitively inadequate and developed their own model with five topological relations that they claimed are better suited to user requirements. The model allows for the incorporation of objects of one or two dimensions. If we follow a recent proposal by Kurata and Egenhofer (2007; 2009) formalizing the relation between a trajectory and a region, we have to distinguish 26 primitive relations. The authors discuss approaches to reduce this number (e.g., Wang, Luo, and Xu 2004, see also Klippel in press). These are only a few examples of developments of topology-based calculi that claim (some more, some less) cognitive adequacy. In several of them, the importance of behavioral user studies is pointed out, but follow up studies are often missing. In this paper, we discuss five experiments that we designed to shed light on three neglected questions of how domain semantics influences the role that topology plays as a tool to model the cognitive conceptualization of movement patterns, whether topological relations are equally salient in different domains, and what cognitively adequate topological invariants of movement conceptualization are. Experiments Over the past five years we have developed a research framework that allows for the assessment of the category construction (conceptualization) of geographic (and other) movement patterns, that is, geographic events. This framework consists of a variety of tools that allow for conducting behavioral research more efficiently to proof, test, or augment the cognitive adequacy of formal spatial calculi. Our approach extends work by Mark and Egenhofer (e.g., Mark and Egenhofer 1994a; Shariff, Egenhofer, and Mark

10 1998) and Knauff and collaborators (Knauff, Rauh, and Renz 1997) by focusing on dynamically changing spatial relations in contrast to static stimuli used in the above mentioned experiments. The core of our research framework are methods and tools combining efficient experimental data collection and visual analytics: CatScan: A tool to administer category construction experiments. KlipArt: A visual analytics environment to explore in depth category construction as well as linguistic behavior (complementing classic analysis techniques such as cluster analysis). MatrixVisualizer: A tool to visualize raw similarity values and a modification of the Levenshtein distance to reveal individual differences. The experiments that are the focus of this paper are a set of five new experiments (rerunning the hurricane experiment Klippel and Li 2009 to match the number of animated icons) that will allow for the in-depth exploration of the relationship between topology and domain semantics (see Section Materials and Figures 1 and 2 for details). The five different domains/scenarios in this series of experiments all have a moving entity and a reference entity (figure and ground Talmy 2000) that are spatially extended: Hurricane / Peninsula (abbreviated as: Hur) Tornado / City (Tor) Ship / Shallow water (Shi) Cannonball / City (Can)

11 Two geometric figures (GeoT) Participants 131 participants took part in five experiments (five scenarios, see above). The goal was to have an equal number of 20 participants in each scenario. What we needed to make sure was that participants were focusing on the domain semantics of each scenario. To this end, we checked the linguistic descriptions of participants (that participants provided at the end of a grouping task, see below) and replaced participants that did not make explicit reference, or made the wrong reference, to the scenario semantics. Additionally, data for one participant was accidentally overridden and was replaced. Participants were undergraduate Penn State students from various disciplines. They were reimbursed $10 for their participation. Details for each scenario are as follows: Hurricane: 9 female, average age Tornado: 9 female, average age Ship: 9 female, average age Cannon: 5 female, average age Geometry: 4 female, average age Material

12 As we are interested in how movement patterns across different semantic domains are conceptualized and how this conceptualization can be captured by formal qualitative theories of representing and reasoning about spatial relations, we followed a design from our previous experiments (Klippel and Li 2009). This design is inspired by the endpoint hypothesis (Regier 1996; Regier and Zheng 2007) and distinguishes movement patterns on the basis of the topological relation in which they end. The basis for distinguishing topologically defined ending relations is a conceptual neighborhood graph (Freksa 1992) that is based on eight topological relations identified by both Egenhofer s intersection models (Egenhofer and Franzosa 1991) and the region connection calculus, RCC (Randell, Cui, and Cohn 1992). As we employ primarily real world scenarios, constraints (such as which entity is moving) are inherent in the domain and experimental design, respectively. We briefly introduce the distinctions we make using the hurricane scenario (see Figure 2). ===== Figure 2 ====== A hurricane approaching and crossing a peninsula progressively changes its spatial relation with the peninsula. While the movement continuously changes Euclidean information such as the distance between the hurricane and the peninsula, the qualitative changes that occur between hurricane and peninsula are often considered as being critical (Hayes 1978; Egenhofer and Al-Taha 1992). Qualitative changes can be captured using topological calculi such as RCC and IM detailing all possible relations between two

13 spatially extended entities. Additionally, we know from research in cognitive science that for conceptualization of events, the ending relation of an event is of particular importance (Regier and Zheng 2007). We designed our experiments around these two aspects, that is, hurricanes (and all other moving entities in our five scenarios) would start in the upper right corner of an animated icon (see Figure 1 and 2) and would make their way across a reference object (e.g., peninsula). The movement patterns are primarily distinguished on the basis of topological equivalence classes. The equivalence classes are based on the conceptual neighborhood graph depicted in Figure 1 and 2. Hence, the resulting ending relations and associated paths through the conceptual neighborhood graph are (using the hurricane scenario as an exemplar): DC1 the hurricane does not make landfall (CNG path: DC; Figure 2a); EC1 the hurricane kind of bumps into the peninsula (CNG path: DC-EC; Figure 2b); PO1 the hurricane just reaches land such that half of the hurricane is on land and the other half is over water (CNG path: DC-EC-PO; Figure 2c); TPP1 the hurricane makes landfall but is still connected to the water (CNG path: DC-EC-PO-TPP; Figure 2d); NTPP the hurricane makes landfall and is completely over land (CNG path: DC-EC-PO-TPP-NTPP; Figure 2e); TPP2 same as TPP1 but the hurricane nearly made it out to the water again (CNG path: DC-EC-PO-TPP-NTPP-TPP; Figure 2f);

14 PO2 same as PO1 but on the other side of the peninsula (CNG path: DC- EC-PO-TPP-NTPP-TPP-PO; Figure 2g); EC2 same as EC1 but on the other side of the peninsula (CNG path: DC- EC-PO-TPP-NTPP-TPP-PO-EC; Figure 2h); DC2 same as DC1 but has crossed the peninsula completely (CNG path: DC-EC-PO-TPP-NTPP-TPP-PO-EC-DC; Figure 2i). Within each topologically defined equivalence class, eight instances of a movement pattern were realized by randomizing start and ending coordinates (without violating topological information). This resulted in a total of 72 animations per scenario (hurricane, tornado, ship, cannonball, geometry) for the nine paths/ending relations as shown in Figure 2 and detailed above. As shown in Figure 1, the actual layout of the icons was kept as similar as possible. Likewise, we ensured that the start and ending coordinates were as similar as possible across the different scenarios, and that the speed was identical across the different conditions. The original set of starting and ending coordinates was created using numbers from the website random.org. Adobe Flash CS4 was used to create the animations and then exported in animated GIF format. The animations were further smoothed and enhanced in appearance using the Easy GIF Animator software. In total, 360 animated icons were created. Design and Procedure

15 All experiments detailed above were group experiments. A GIS lab in the Department of Geography at Penn State was equipped with view blocks such that up to 16 participants could participate in one experimental session at the same time but were not able see each other s screens. Each workstation was equipped with a 24 Dell wide screen monitor, which provided optimal conditions for performing a grouping experiment on a screen in that nearly all icons were initially visible. The grouping software CatScan (inspired by a tool Knauff and collaborators Knauff, Rauh, and Renz 1997 designed) was used for the participation portion of the experiment. The participants were randomly assigned to computers, provided consent, and entered anonymous personal data such as age and field of study. They received a short introduction detailing the scenario explicitly and the course of the experiment. This text explicitly referred to the semantics of the stimulus, for example, hurricane and peninsula. The instructions also made it clear that there were no right or wrong ways to create groups (categories), and that it was up to the participants themselves to select criteria and the number of appropriate groups (categories). To familiarize participants with the grouping software, participants were required to complete a warm-up task in which they were asked to group animals such as dogs, cats, and camels. Participants had to place all icons into groups before moving on to the main experiment. Participants were presented with the 72 animated icons of one of the five scenarios, that is, a participant performed the grouping task once with only one scenario. In the first part of the main experiment, participants created as many groups (categories) as they deemed appropriate. Participants had to explicitly create all groups, and no groups were depicted or implied at the start of the experiment (see the top half of Figure 3).

16 Participants can create and delete groups, move animated icons into and out of groups, or move icons between groups. This procedure is referred to as category construction (Medin, Wattenmaker, and Hampson 1987), free classification, or unsupervised learning (Pothos and Chater 2002). ======= Figure 3 ======= After placing all icons into groups, participants entered the second part of the main experiment. In this part, participants were presented again with the groups that they had created in the first part. The groups were presented one at a time, and the participants were asked to linguistically label the groups. The linguistic labeling task had two parts: a) to provide a short description of no more than 5 words, and b) to provide a more detailed explanation of the rationale upon which a particular group was created. Results The experiment software, CatScan, collects rich data about the grouping (category construction) behavior of participants: Which icons are placed into groups together. The number of groups per participant.

17 The time participants needed to perform the grouping task. The order in which icons are placed into groups. Linguistic labels of the groups created. This rich dataset allows us to create a comprehensive picture of participants' grouping behavior. To further enhance our understanding of grouping behavior, we are using a variety of custom-made visual analysis tools that complement more traditional data analysis methods. In the following we will detail: Basic statistics such as comparing number of groups and grouping time. A qualitative analysis using visualized similarities. Assessment of the overall conceptual structure of individual scenarios based on the similarity ratings of icon pairs, using cluster analysis and multidimensional scaling. A statistical analysis using chi-square data derived from visual analysis using KlipArt. Complementary linguistic analysis. Preliminaries The following categorization of topological ending relations (derived from RCC) will be important for the discussion: Relations that do not overlap (DC1, EC1, DC2, and EC2), relations that do overlap (PO1 and PO2), and relations that are characterized as proper part (TPP1, NTPP, and TPP2).

18 Basic statistics number of groups and grouping time Figure 4 shows box plots of the number of groups that participants created for the five different scenarios. The data across the five scenarios are comparable with one exception: In the hurricane scenario one participant created 29 groups. ANOVA supported the informal interpretation that statistically there is no difference in the number of groups that participants created (with or without the outlier). ===== Figure 4 ====== Figure 5 shows the time that participants spent to group the icons in each scenario. The interpretation leads to conclusions similar to those that we made regarding the number of groups, that there are no statistically significant differences between the five different scenarios. ANOVA confirmed this interpretation. Interestingly, the hurricane scenario has two outliers: Participant 11, who created 29 groups and participant 10. ====== Figurer 5 ======= Grouping data Similarity The category construction of each participant resulted in a 72 x 72 similarity matrix. 72 is the number of icons we used. This matrix encodes all possible similarities between pairs of icons at a time for all icons in the data set, similar to a spatial weights matrix; it is a

19 symmetric similarity matrix with 5184 cells. Similarity is encoded by a 0-1 binary; any pair of icons is coded as 0 if its two icons are not placed into the same group, and 1 if its two items are placed into the same group. The overall similarity of two items is obtained by summation over all the similarity matrices of individual participants. For example, if two icons (called A and B) were placed into the same group by all 20 participants, we add 20 individual 1 s to obtain an overall score of 20 in the respective cells for matrix position AB and BA. MatrixViewer A tool we developed, MatrixViewer (Klippel, Weaver, and Robinson 2011), allows visualization of the raw similarities for all icons at the same time. Figure 6 shows these matrices for the five scenarios. The topological equivalence classes (i.e., different ending relations) are blocked and labeled for the ship scenario. Each block contains eight icons with the same topological information. We see that along the diagonal the blocks are dark red. This is a first indication that participants used topological information to form categories, because the blocks are animated icons with the same topological information. However, even within these blocks that show participant behavior for each topological equivalence class, we find differences across the different scenarios (compare, for example, cannon and geometry). Even more differences can be found when comparing pairs of topological equivalence classes. While some scenarios show higher similarities for topological relations that have proper part relations (e.g., hurricane and geometry), others show higher similarities for non-overlapping relations (e.g., cannon). Certain

20 similarities are unique for individual scenarios such as higher (darker) similarities for partial overlap relations in the geometry scenario. These matrices provide a first overview of how topological relations might be differently conceptualized across different semantic domains. We provide detailed analysis of different aspects of the conceptualization of movement patterns in the following sections. ====== Figure 6 ====== Cluster analysis To analyze the cognitive conceptualizations of movement patterns across the five scenarios, we used the overall similarity matrix. To this end, we subjected the overall similarity matrix to a hierarchical cluster analysis using the software CLUSTAN. We compared different cluster methods as a way to validate our interpretations: Ward s methods, average linkage, and complete linkage. Figure 7 shows the results for Ward s method 1. There are a couple of important observations to make regarding the category structure that the cluster analysis reveals. For all scenarios we find that the saliency of topologically defined ending relations varies. This is in contrast to research on static spatial relations in experiments by Knauff and collaborators (Knauff, Rauh, and Renz 1 We have created a website that contains additional documentation of the results (

21 1997), who found that static topological relations are salient to the same degree. These results are in line with research that asserts that there is a superstructure present that groups topological and corresponding qualitative temporal relations (e.g., Lu, Harter, and Graesser 2009). We will come back to this aspect in the discussions section. More importantly, and unique to our experimental setup, we find that across different semantic domains (across the five scenarios) the grouping structure changes. We will first describe this change using results from cluster analysis (We have supplementary results from multidimensional scaling on the supporting website, compare also the direct visualization of the similarity values using MatrixVisualizer in the previous Section). ===== Figure 7 ====== Overall, in all scenarios, topologically defined ending relations play an important role. This is revealed by the grouping behavior of participants in that topological equivalence classes are at the bottom of the dendrograms with the highest similarity values. We also found that certain topological ending relations form groups, and that these groups are not identical across different scenarios. In the hurricane scenario we found that EC1 and PO1 are conceptually very close, and DC1 joins this group conceptually somewhat later. Clearly distinguished from this group are equivalence classes whose ending relations are proper parts, TPP1, NTPP, and TPP2. Hurricane movement patterns that (nearly) cross the peninsula are again close together, such as DC2 and EC2; PO2 joins this group conceptually somewhat later. In the tornado scenario we also found three main groups.

22 This time, however, the relations that did overlap are separated from those that did not, DC1/EC1, DC2/EC2, versus PO1, TPP1, NTPP, TPP2, PO2. In the ship scenario, we found a pattern that combines aspects of both previous scenarios. On the one hand, we have a group of EC1/PO1 and DC1 joins this group conceptually somewhat later, but for the movement close to crossing the shallow water we find a clearer distinction between overlapping and non-overlapping relations (PO2 joins the proper part relations, not the DC2/EC2 group). The cannon scenario shows a very clear pattern: non-overlapping relations (DC1/EC1 and DC2/EC2) are separated from overlapping and proper part ones (PO1, TPP1, NTPP, TPP2, PO2). Last but not least, the geometry scenario (GeoT) has a unique pattern. The partial overlap relations are singled out as a group (PO1/PO2) which did not occur in any other scenario. DC1/EC1 are clearly grouped together as are DC2/EC2, and proper-part relations (TPP1, NTPP, TPP2) are once again in their own group. Grouping data raw frequencies To add to the analysis and reveal the statistical significance of the results discussed in the previous sections (raw similarities, cluster analysis) we used one of our custom visual analysis tools, KlipArt (Klippel, Weaver, and Robinson 2011). This tool offers an interactive analysis environment in which grouping behaviors of individual participants, individual icons or groups of participants/icons can be assessed. Figure 8 shows part of the interface. The numbers in yellow squares are participants, and the three digit numbers

23 are identifiers for animated icons (which are visible in the interface but not displayed here.). ===== Figure 8 ====== With this tool it becomes possible to elicit numerical data suitable for chi-square analysis. As a first step in this analysis, we looked into the grouping behavior of participants in relation to the nine topologically defined ending relations (DC1, EC1, PO1, TPP1, NTPP, TPP2, PO2, EC2, DC2). By selecting icons of one topological equivalence class (for each of the five scenarios) we quickly found the number of participants that placed all these icons into the same group. Additionally, by looking into the linguistic descriptions that participants provided, we revealed the grouping rationale of participants. ====== Figure 9 ======= The first question we addressed is whether there are statistically significant differences between the numbers of topologically equivalent movement patterns that are placed into the same group within each scenario. To this end, we performed a chi-square analysis that compared the nine topological equivalence classes within each scenario. Interestingly (see Figure 9), only the cannon example yielded a significant difference

24 between the different topological equivalence classes (Χ = , df = 8, p <.001). We followed up on these results by looking into the linguistic descriptions that participants provided, which we will discuss in more detail below. The second question addressed the same topological equivalence class across different scenarios. How do the numbers of participants who placed all icons belonging to a particular topological equivalence class into the same group compare across different scenarios? We used the data collected from the KlipArt analysis to make this comparison. For each topological equivalence class across the five different scenarios, we compared the number of participants who placed all icons that belong into the selected equivalence class into the same group. Chi-square analysis revealed that there was no significant difference comparing each topological equivalence class across the five scenarios. The only exception was a significant difference for the ending relation EC2 (Χ = , df = 4, p =.03). A look into the standardized residuals showed that this effect is most like caused by the ship scenario with a z score of 1.7; a screenshot of the grouping behavior for EC2 in the ship scenario is shown below. We followed up on these results by looking into the linguistic descriptions of participants (see below). The difference between the ship scenario EC2 grouping behavior and those for other topological equivalence classes is that we found a large number of participants who grouped All-but-one animation of this equivalence class together (see grey box in Figure 10). If we relax the criterion that participants have to place all icons of a topological equivalence class into the same group to all-but-one, we do not find a statistical difference in the comparison of topological ending relations. ======

25 Figure 10 ====== The third question addressed how the similarity of conceptual neighbors changes from a statistical perspective. We have already seen in the cluster analysis (see Section Cluster Analysis) that certain topological equivalence classes naturally group together, and that this natural grouping is different for semantically different scenarios. Here, we add to this analysis by comparing the numbers of participants who placed all icons of two topological equivalence classes, which are conceptual neighbors, together into the same group. In other words, topological relations that are connected by an edge in the conceptual neighborhood graph (see Figure 2) as the translation movement of the moving entity determines which topological relation(s) can be reached from each other. Figure 11 shows the data we extracted from KlipArt. The figure shows some interesting patterns with scenario-relevant differences. The overall pattern shows three peaks: the first peak is formed by DC1 and EC1 ending relations and shows that many participants placed these to topological equivalence classes into the same category; the second, wider peak shows participants who placed ending relations that are proper part relations into the same category (pairs: TPP1 and NTPP; NTTP and TPP2); the third peak shows EC2 and DC2 ending relations. Figure 11 therefore reinforces the general trend of the cluster analysis. Almost all scenarios exhibit a higher similarity of relations that are proper part relations (TPP1/NTPP/TPP2). The conceptual neighborhood similarities, however, are different across the five scenarios. While some scenarios make a clear distinction between nonoverlapping (DC1, EC1, DC2, and EC2) and overlapping (PO, TPP, NTPP) relations, others make a clearer distinction between proper part relations and others. There also

26 seems to be a difference of similarities (in some scenarios) between conceptual neighbors depending on the dynamic context: whether two relations are conceptual neighbors at the beginning or at the end of a movement pattern. To add a statistical analysis to this interpretation, we again used the KlipArt tool to extract the number of participants who placed all pairs of topological equivalence classes, where the ending relation is a conceptual neighbor, into the same group. The expected values for cells in the chi-square table are at times are below 5 for more than 20% of the cells, which is problematic from a statistical perspective (Field 2009). Siegel and Castellan (1988) discussed a solution to this problem suggesting to combine categories where appropriate. In our case, the formal basis of our experiments offers the following aggregation of data. This distinction works similarly, although not identically, for RCC and the intersection models. We combined the two DC/EC pairs (DC1&EC1 with EC2&DC2), referred to as non-overlapping relations, the pairs that entail partial overlap (EC1&PO1 with EC2&PO2 with PO1&TPP1 with TPP2&PO2), referred to as overlapping relations, and pairs of total overlap, the proper part relations, (TPP1&NTPP with NTPP&TPP2). Please note that while participants were required to have placed two conceptual neighborhood relations together (see Figure 2), they were not required to have placed all relations involved in these pairs together. We performed again two types of analysis: first, comparing the three aggregations within the same scenario; second, comparing the same aggregation across the five scenarios. In the first Chi-square analysis we looked into differences within each scenario with respect to the three aggregates. For each scenario we compared the numbers of nonoverlapping, overlapping, and proper part relations. We found statistically significant

27 differences for all but the tornado scenario (Hurricane: Χ = , df = 2, p =.001; Tornado: Χ = 2.702, df = 2, p. >.05 ; Ship: Χ = 9.351, df = 2, p =.008; Cannon: Χ = , df = 2, p <.001; GeoT: Χ = , df = 2, p <.001). These results show that there is in almost all cases a significant difference in the similarities of conceptual neighbors which indicates that some conceptual neighbors are conceptually closer than others. We follow up on the non-significant differences in the tornado scenario in the linguistic analysis. The second Chi-square analysis addressing differences across scenarios showed that non-overlapping and proper-part topologically defined ending relations are statistically significantly different across scenarios (non-overlapping: Χ = , df = 4, p =.003; proper part: Χ = , df = 4, p =.009), while overall the differences for overlapping relations did not turn out to be significant. The lack of statistical significance is most likely associated with overall rather low number in this case. Additionally, it should be pointed out that for some scenarios there are noteworthy differences, such as PO1&TPP1 and TPP2&PO2, respectively, are never grouped together in the geometry scenario, and in the tornado, hurricane and cannon scenarios PO2 and EC2 are never grouped together (taking into account all 16 icons). Overall, this means that the results of the cluster analysis, visualized by the dendrograms, have a statistical basis. Conceptual neighbors are not equally similar to each other especially in the case of non-overlapping (DC and EC) and proper-part relations. ======

28 Figure 11 ======= For the last part of this analysis, we looked into the grouping behavior of participants for identical topological ending relations that are contextually different. We compared ending relation DC1 with ending relation DC2, ending relation EC1 with ending relation EC2, and so forth. Figure 12 provides an overview of the raw numbers. In general, we find a trend that relations that are closer with respect to the length of the conceptual path between (in the CNG) them are also rated as being closer by the participants. Pooled numbers for DC1/DC2, EC1/EC2, PO1/PO2, and TPP1/TPP2 are 16, 18, 21, 49, respectively. ====== Figure 12 ====== Compared to scenarios from the semantic domains, the geometry scenario consistently shows the highest numbers. If no meaning is attached to the moving and reference entity, it appears to be an option to consider them as conceptually similar even though they are not conceptual neighbors. For all scenarios (except the cannon one) the two TPP relations show the highest similarity. This is consistent with the similarity that one would derive from the conceptual neighborhood graph, that is, these relations are only separated by NTPP. Given the relatively small numbers we did not perform a statistical analysis.

29 Linguistic analysis Our goal with the linguistic analysis was to shed light on the grouping behavior (in contrast to being a fully laid out linguistic analysis). To this end, we used KlipArt to select grouping behaviors that we previously pointed out in the article. KlipArt allows for directly assessing the linguistic descriptions associated with the grouping behavior. The first analysis is addressing the non-significance of comparing nonoverlapping, overlapping, and proper part relations of the tornado scenario as discussed above. To this end, we focused primarily on the proper part relations, which seem to yield low numbers compared to other scenarios such as the hurricane one. However, the cannon scenario yields the lowest number for proper part relations, so we included it here as well. It is important to note that both scenarios share the same reference entity of a city. We will discuss both scenarios together, because the reasons for the more differentiated view of proper part relations are similar. Looking through the descriptions, we found, a large number of participants who made a point to separate different parts of the city (the reference entity). In the tornado scenario we found linguistic descriptions (for the short labels) for proper part relations (TPP1, NTPP, TPP2) such as: Inside upper, inside lower, upper side, east, west, bulls eye tornado, southwest corner. Interestingly, we found a similar pattern for the cannon example: Many participants differentiated the locations in the city where the cannon ball hit in the same way as in the tornado scenario. For both scenarios, this brief analysis shows that some reference entities (potentially in

30 combination with the moving entities) lead to a more fine grained distinction of the topological proper-part relations. This also explains the statistically significant difference we found in the cannon example. While the topological relations DC1/EC1 and DC2/EC2 (the city is not hit) form a very strong conceptual bond, participants differentiated very strongly which part of the city got hit (particularly the proper part relations). In the ship scenario we also found that information other than topology is at least partially responsible for the higher degree of differentiation. For the EC2 ending relation that we discussed above, we found that participants used direction information to differentiate trajectories of ships, which at least partially explains the lower number of participants who placed all icons of EC2 into the same group. However, as we randomized direction as a by-product of randomizing start and ending coordinates, the emerging pattern is not unanimous. Discussion Our experiment findings reveal several important aspects of the conceptualization of geographic movement patterns. The first to note is that for individual topologically defined ending relations, we found that topology is indeed a fairly strong candidate to predict (and model) the cognitive conceptualization of movement patterns. However, this is not the case for all scenarios and all topological relations; it figures most prominently in the proper part relations in the cannon scenario and to a lesser extent in the tornado scenario. Additionally, even in scenarios in which topology is the primary means to guide

31 grouping behavior, we found that other aspects such as direction (e.g., as discussed for the ship scenario) play an important differentiating role. In the cannon and tornado scenarios we used the same reference entity, a city. One reason for the less dominant role of topology may lie in the greater differentiation that a city offers in comparison to, for example, a peninsula or the area of shallow water in the ship scenario. It is certainly not a trivial distinction where a city gets hit and this may change the focus of conceptualizing geographic movement pattern from using topological distinctions to other information available. This aspect is even more astonishing as we did not systematically vary the location within a topological ending relation. We randomized the ending coordinates but did not stratify them, for example, by dividing the city into four parts and equally spreading the ending coordinates across those four parts. The important aspect here to summarize is that certain topological relations do elicit a fine within distinction (that is, topology may not be the primary conceptual glue) in combination with a certain reference entity while others do not. The DC1 and DC2 relations, for example, for a hurricane (which does not hit the peninsula) or a cannon ball (which does not hit a city) are stronger predictors of category membership than proper part relations. Shifting the focus of the discussion to the relation between topologically defined ending relations, that is, neighbors in the conceptual neighborhood graph, we found that for cognitively conceptualizing movement patterns, pairs of topological relations are not equally salient across different semantic domains. While certain relations are frequently grouped together across different semantic domains, others are influenced by the semantics of the domain. Most strikingly, a general distinction is made between overlapping and non-overlapping relations with the partial overlap relation (PO) being

32 more volatile. This supports results from early experiments by Mark and Egenhofer (e.g., Mark and Egenhofer 1994a), who found in their manifold experiments that certain topological relations are conceptually closer than others (for a single semantic domain, i.e., a road crosses a park, see also Klippel and Li 2009). In contrast, findings by Knauff and collaborators (Knauff, Rauh, and Renz 1997) are not replicated, neither in the geometric scenario nor in the semantic ones, as Knauff s experiments using static spatial relations showed that participants distinguish topological relations identified by RCC-8 or Egenhofer s intersection model more or less equally. However, the most important aspect that our data clearly shows is that the domain semantics (i.e., the semantic context) plays a role in assessing the conceptual similarity between topological relations. This showed up first in visualizing the raw similarity values using a matrix visualization approach. While topological ending relations are certainly important (the groups along the diagonals), the overall pattern changes from scenario to scenario. These differences show up in the cluster analysis for each of the scenarios. We can make two important observations here: First, the similarity between topological relations changes across different scenarios; second; there may be differences (in some scenarios) between the same topological relations in dependence of whether they are at the beginning or end of a movement pattern (e,g., DC1/EC1/PO1 versus DC2/EC2/PO2). One interesting distinction we can make is that the geometry scenario (with no explicitly introduced semantics) relies most heavily on topological information with one additional peculiarity. In no other scenario do we find a violation of the general similarity introduced by the conceptual neighborhood graph. Only in the geometry scenario did participants place PO1 and PO2 ending relations together into the same group such that

33 the cluster analysis considers them as being a group by themselves. This would indeed require a reorganization of the conceptual neighborhood graph, for example, by introducing additional edges (Worboys and Duckham 2006). All other scenarios, as different as the conceptual similarities may be, do not violate the CNG structure, but certainly require a reassessment of the weights of the edges in the graph (see below). Given the differences in how topologically defined ending relations are grouped together, it is therefore worthwhile to look into aspects of granularity and how different levels of granularity are handled in the two formalisms (intersection models and RCC). Figure 13 shows the two most prominent levels, that is, the level of eight and five distinguished topological relations. The two models both predict that the two proper part relations are conceptually closer, that is, on the coarse level of granularity no distinction is made between NTPP and TPP. This prediction corresponds largely to the findings in our experiments across all scenarios. Most prominently, we find that in the hurricane, ship and geometry scenarios, proper part relations are considered as being conceptually close. And, even in the cannon and tornado scenario for which we find that proper part relations are more differentiated, we still see that overall those relations form a conceptual group (together with partial overlap relations, as shown in the cluster analysis). In contrast, the two models (intersection and RCC) differ in the way they handle the granularity shift for the relations DC, EC, and PO. While the region connection calculus groups DC and EC together, the intersection models combine EC and PO. Our results correspond to this mismatch in that we find that this distinction is dependent on the domain semantics. While in the tornado, cannon, and geometry scenarios participants

34 clearly favor the conceptual closeness of DC1 and EC1 relations, in the hurricane and ship scenarios they have a tendency to group EC1 and PO1 together more often. This makes sense if one takes into account the characteristics of each domain that influence grouping behavior. It is important to note, though, that both the hurricane and the ship scenarios make a distinction with respect to the length of the path through the conceptual neighborhood graph (DC1/EC1 versus DC2/EC2). In case the hurricane (ship) moves toward the peninsula (shallow water), DC1 is separated from EC1 and PO1. In cases in which the hurricane (ship) has made it across the peninsula (shallow water) DC2 and EC2 are considered as being conceptually closer. ====== Figure 13 ====== There are several approaches in the literature that discuss how topological relations should be grouped together, with the goal of providing a cognitively more adequate characterization of spatial relations (e.g., Clementini, Di Felice, and van Oosterom 1993). To the best of our knowledge, this article is the first that systematically compares different semantic domains with respect to their influence on how formally defined topological relations correspond to cognitive conceptualizations of spatial relations. We are in a position to modify assumptions made in the literature (Hernández 1994; Bruns and Egenhofer 1996; Li and Fonseca 2006; Camara and Jungert 2007) on how similar topological relations are, with the additional requirement of taking into account the semantics of a particular domain. We can do this in a qualitative way as

35 discussed above, using the different levels of granularity identified in topological calculi (based on actual user tests). Additionally, we are exploring options to more quantitatively specify the similarity between topological relations by weighting the edges in the conceptual neighborhood graph. This approach extends work by Li and Fonseca (Li and Fonseca 2006) in two ways: first, our assumptions are based on behavioral assessments and second, we are able to offer weights that are domain-specific. The latter aspect is a work in progress, and we only briefly show how our behavioral results can be applied as weights in a conceptual neighborhood graph (see Figure 14). To this end, we use the results of the cluster analysis that we discussed in Figure 7. Cluster analysis provides fusion coefficients, a quantitative value that indicates the conceptual distance (similarity/dissimilarity) at which two animated icons (in the case of our experiments) are joined together. As we have seen, animated icons with one topological equivalence class show across all scenarios a tendency to be conceptually very close, so that we can use topological equivalence classes instead of individual animated icons. The numbers shown in Figure 14 are directly derived from the cluster analysis for movement patterns from DC1 to NTPP (but not back, that is, NTPP to DC2). These values quantitatively reflect the various discussions we have detailed above and will allow for modifying weighting schemes for conceptual neighborhood graphs based on user experiments and for specific domains. ====== Figure 14 =======

36 A last aspect we would like to point out is that our results support current theories in cognitive science on event segmentation. Zacks (2004) developed a model to explain how observers perceive ongoing activities. Figure 15 shows this model. Zacks pointed out that to understand how events are understood and, in his experiments, are segmented, one has to take into account both sensory input (bottom-up) as well as knowledge structures (top-down). Our experiments bear resemblance to research on event segmentation in that the topologically defined ending relations offer breakpoints, and by grouping breakpoints together, participants make a choice which ones they consider as more or less differentiating. Our results support Zack s model as they show that the geometry scenario (with no explicit knowledge structures) leads to the finest grained distinctions, as would have been predicted in his model (bottom-up input leads to fine grained distinctions). ===== Figure 15 ====== Conclusions and Outlook The most important aspects of our research results are the qualitative and quantitative assessment of similarities between topological relations based on behavioral data. Grounding (Harnad 1990) formal relations this way improves the often claimed cognitive adequacy of qualitative (here topological) calculi (Egenhofer and Mark 1995a; Renz 2002; Cohn and Renz 2008; Matsakis, Wawrzyniak, and Ni 2010). From our findings we are able to formulate two take home messages:

37 - Topology is important to formally characterize the cognitive conceptualization of movement patterns; yet, not all eight topological relations (as identified by RCC/IM) are equally salient. - The salience of topological relations changes dependently of a semantic domain. In more detail, the importance of the presented research lies in the following: First, we started to thoroughly explore the role that topology plays across different semantic domains, specifically, movement patterns of entities in geographic space. We found both similarities and differences across different domains. From a general perspective, we can claim with some certainty that the part of relationships (TPP and NTPP) are conceptually close in case of movement patterns, and that non-overlapping relations are separated from overlapping ones. Second, we showed that the different conceptual distances between topological relations are an important aspect that needs to be taken into account in cases where conceptual neighborhood graphs are used to model the similarity of geographic events (e.g., Camara and Jungert 2007; Schwering 2008). We are not aware of approaches that use different weights for edges in the conceptual neighborhood graph based on domain semantics. However, our results indicate that such weighting is important to tweak the cognitive adequacy of conceptual neighborhood graphs if they will be used as a tool to model spatial cognitive processes. Third, part of our contribution lies in the development of efficient tools for conducting behavioral research and analyzing behavioral data. Of particular importance is that computational developments allow for presenting animations and thereby directly

38 assessing the conceptualization of movement patterns. We also developed new ways of analyzing behavioral data. Using a visual analysis framework (Weaver 2004; Klippel, Weaver, and Robinson 2011), we have created tools for in-depth conceptual and linguistic analysis of behavioral data. We consider this an essential step in allowing more researchers in the spatial sciences to efficiently conduct behavioral experiments. We are additionally working on more ways of similarity assessment and on improving the linguistic analysis of our data (e.g., using the UAM corpus tool). One of the important findings of our research is that the role of topology in concert with other aspects of spatial information is still not fully understood from a cognitive perspective. To continue revealing the relationship between formal theories in information science and how spatial information is conceptualized, we have designed several studies that complement the research reported in this paper which we consider as essential: 1. Different topological transformations. Dynamics in geographic space can take many forms. The experiments reported in this paper, and the previous experiments we and others have conducted, all address translation movements. In contrast, many geographic events show other or additional dynamics such as scaling (Egenhofer and Al-Taha 1992; Yuan and Hornsby 2008; Jiang and Worboys 2009). To this end, we have designed follow up experiments with scenarios that exhibit extension and shrinking, rather than translation. Candidate scenarios are oil spills, flooding, and desertification. It is important to note that topologically scaling movement patterns can be identical to the translation patterns we used. The question will be whether the same topologically defined ending relations are

39 important in scaling scenarios, and which adjustments have to be made to achieve cognitive adequacy. 2. The direct comparison between static and dynamic icons. It is one of the most pertinent questions in knowledge representation and associated areas such as cartography and visual analysis: how does a specific representation of information/knowledge influence the way that information is cognitively and perceptually processed (MacEachren 1995; Garlandini and Fabrikant 2009)? In an attempt to answer this question, we have completed a short experiment in which we created icons that show paths of hurricanes with a hurricane symbol indicating the ending relation (Li, Klippel, and Yang 2011). The icons and the spatial information represented in them is identical to their dynamic counterparts except for being static. We found two interesting aspects: The first is that ending relations are emphasized when information is presented statically. These findings, and in comparison to the results and experimental setup by Knauff and collaborators (Knauff, Rauh, and Renz 1997), stress the different role of topology in static and dynamic settings. Second, participants used different linguistic means to characterize the groups that they created. For example, while the word path is not used in the dynamic condition, it occurs frequently in the static one. It will therefore be important to run experiments with even more realistic stimuli to disentangle the influences of different forms of representations, especially for representing spatio-temporal information. 3. Evaluating additional calculi. Based on recent work by Kurata and Egenhofer (2009), we ran a pre-study testing the 26 relations that are distinguished on the

40 basis of the 9+ intersection formalism (Kurata and Egenhofer 2007), that is, the relation between a directed line (Dline) and a region. The advantage of this calculus is a greater flexibility in distinguishing between starting and ending relations. While we started with a geometric realization, we are working on real world examples for this case. 4. The complexity of our experiments is restricted to two entities, a moving entity (figure) and a reference entity (ground). In collaboration with researchers from the SFB/TR-8 in Bremen, we are working on studies that involve several entities. As many real world situations may take into account more than two entities, we consider this research direction essential for allowing for cognitively grounding similarity assessments for more complex scenes (see Bruns and Egenhofer 1996; Li and Fonseca 2006). Acknowledgements I would like to thank the reviewers for valuable feedback. I am particularly grateful to Christian Freksa for comments on an earlier draft. I would like to acknowledge Frank Hardisty, Chris Weaver, Rui Li and Jinlong Yang for their help collecting data and creating wonderful software solutions. I also would like to acknowledge Thilo Weigel who implemented the grouping tool that Markus Knauff and collaborators used, which inspired our grouping tool. I sincerely thank Stefan Hansen for implementing our first grouping tool. Research for this paper is based upon work supported by the National Science Foundation under Grant No

41 References Allen, J. F Maintaining knowledge about temporal intervals. Communications of the ACM 26 (11): Bowerman, M Containment, support, and beyond. Constructing topological spatial categories in first language acquisition. In The categorization of spatial entities in language and cognition, ed. M. Aurnague, L. Vieu, and M. Hickmann, Amsterdam: Benjamins. Bruns, H. T., and M. J. Egenhofer Similarity of spatial scenes. In Seventh international symposium on spatial data handling (SDH'96), Delft, The Netherlands, ed. M. J. Kraak and M. Molenaar, Cai, G. forthcoming. Contextualization of geospatial database semantics for mediating human-gis dialogues. GeoInformatica. Camara, K., and E. Jungert A visual query language for dynamic processes applied to a scenario driven environment. Journal of Visual Languages and Computing 18: Carlson, L. A., C. Hölscher, and T. F. Shipley, ed Proceedings of the 33rd Annual Conference of the Cognitive Science Society.. Austin, TX: Cognitive Science Society. Clementini, E., P. Di Felice, and P. van Oosterom A small set of formal topological relationships suitable for end-user interaction. In Advances in Spatial Databases. Third International Symposium, SSD '93 Singapore, June 23 25, 1993 Proceedings, ed. D. Abel and B. C. Ooi, Berlin: Springer. Cohen, R What's so special about spatial cognition? In The development of spatial cognition, ed. R. Cohen, Hillsdale, NJ: Erlbaum.

42 Cohn, A. G., and J. Renz Qualitative spatial representation and reasoning. In Handbook of knowledge representation, ed. F. van Harmelen, V. Lifschitz, and B. Porter. Amsterdam: Elsevier. Davis, E Representations of commonsense knowledge. San Mateo, CA: Morgan Kaufmann Publishers. Dey, A. K Understanding and using context. Personal and Ubiquitous Computing 5 (1):4 7. Egenhofer, M. J., and D. M. Mark. 1995a. Modeling conceptual neighborhoods of topological relations. International Journal of Geographical Information Systems 9 (5): b. Naive geography. In Spatial Information Theory. A Theoretical Basis for GIS. International Conference, COSIT 95, Semmering, Austria, September 21-23, 1995, Proceedings, ed. A. U. Frank and W. Kuhn, Berlin: Springer. Egenhofer, M. J., and K. K. Al-Taha Reasoning about gradual changes of topological relationships. In Theories and methods of spatio-temporal reasoning in geographic space, ed. A. U. Frank, I. Campari, and U. Formentini, Berlin: Springer. Egenhofer, M. J., and R. D. Franzosa Point-set topological spatial relations. International Journal of Geographical Information Systems 5 (2): Fernyhough, J., A. G. Cohn, and D. C. Hogg Constructing qualitative event models automatically from video input. Image and Vision Computing 18 (2): Field, A. P Discovering statistics using SPSS. and sex, drugs and rock'n'roll (3rd Ed). London: Sage Publ.

43 Freksa, C Temporal reasoning based on semi-intervals. Artificial Intelligence 54 (1): Galton, A Qualitative spatial change. Oxford: Oxford Univ. Press. Garlandini, S., and S. I. Fabrikant Evaluating the effectiveness of efficiency of visual variables for geographic information visualization. In Spatial Information Theory, 9th International Conference, COSIT 2009, Aber Wrac'h, France, September 21-25, 2009 Proceedings, ed. K. Stewart Hornsby, C. Claramunt, M. Denis, and G. Ligozat. Berlin: Springer. Gottfried, B., and H. Aghajan, ed Behaviour monitoring and interpretation. Ambient intelligence and smart environments. Amsterdam: IOS Press. Harnad, S The symbol grounding problem. Physica D: Nonlinear Phenomena 42 (1-3): Hayes, P The naive physics manifesto. In Expert Systems in the Microelectronic Age, ed. D. Michie, Edinburgh: Edinburgh University Press. Hernández, D Qualitative representation of spatial knowledge. Berlin: Springer. Hobbs, J. R., and R. C. Moore, ed Formal theories of the common sense world. Norwood, NJ: Ablex. Jackendoff, R., and B. Landau Spatial Language and Spatial Cognition. In Languages of Mind, ed. R. Jackendoff, Cambridge (MA), London: MIT Press. Jiang, J., and M. Worboys Event-based topology for dynamic planar areal objects. International Journal of Geographical Information Science 23 (1):33 60.

44 Johnson, M The body in the mind: The bodily basis of meaning, imagination, and reasoning. Chicago: University of Chicago Press. Klippel, A. forthcoming. Movement choremes. Bridging cognitive understanding and formal characterization of movement patterns. TopiCS. Klippel, A., and R. Li The endpoint hypothesis: A topological-cognitive assessment of geographic scale movement patterns. In Spatial Information Theory, 9th International Conference, COSIT 2009, Aber Wrac'h, France, September 21-25, 2009 Proceedings, ed. K. Stewart Hornsby, C. Claramunt, M. Denis, and G. Ligozat, Berlin: Springer. Klippel, A., C. Weaver, and A. C. Robinson Analyzing cognitive conceptualizations using interactive visual environments. Cartography and Geographic Information Science 38 (1): Klix, F Information und Verhalten. Kybernetische Aspekte der organismischen Informationsverarbeitung; Einführung in naturwissenschaftliche Grundlagen der allgemeinen Psychologie. Bern: Huber. Knauff, M., R. Rauh, and J. Renz A cognitive assessment of topological spatial relations: Results from an empirical investigation. In Spatial information theory: A theoretical basis for GIS, ed. S. C. Hirtle and A. U. Frank, Berlin: Springer. Kurata, Y., and M. J. Egenhofer The 9+-Intersection for Topological Relations between a Directed Line Segment and a Region. In 1st Workshop on Behaviour Monitoring and Interpretation (BMI'07), in conjunction with 30th German Conference on Artificial Intelligence, Osnabrueck, Germany, September 2007, ed. B. Gottfried, Bremen: Universität Bremen.

45 Interpretation of behaviors from a viewpoint of topology. In Behaviour monitoring and interpretation. Ambient intelligence and smart environments, ed. B. Gottfried and H. Aghajan, Amsterdam: IOS Press. Lakoff, G Women, fire and dangerous things. Chicago: Chicago University Press. Li, B., and F. Fonseca TDD: A comprehensive model for qualitative spatial similarity assessment. Spatial Cognition and Computation 6 (1): Li, R., A. Klippel, and J. Yang Geographic event conceptualization. Where spatial and cognitive sciences meet. In Proceedings of the 33rd Annual Conference of the Cognitive Science Society., ed. L. A. Carlson, C. Hölscher, and T. F. Shipley, Austin, TX: Cognitive Science Society. Lu, S., D. Harter, and A. C. Graesser An empirical and computational investigation of perceiving and remembering event temporal relations. Cognitive Science 33: MacEachren, A. M How maps work: Representation, visualization, and design. New York: The Guilford Press. Mandler, J. M How to build a baby II. Conceptual primitives. Psychological Review 99 (4): Mark, D. M Spatial representation: A cognitive view. In Geographical information systems: Principles and applications. Second edition, v. 1, ed. D. J. Maguire, M. F. Goodchild, D. W. Rhind, and P. A. Longley, Mark, D. M., and M. J. Egenhofer. 1994a. Calibrating the meanings of spatial predicates from natural language: Line-region relations. In Advances in GIS Research, 6th

46 International Symposium on Spatial Data Handling, Edinburgh, Scotland, UK., ed. T. C. Waugh and R. G. Healey, b. Modeling spatial relations between lines and regions: Combining formal mathematical models and human subject testing. Cartography and Geographic Information Systems 21 (3): Matsakis, P., L. Wawrzyniak, and J. Ni Relative positions in words: a system that builds descriptions around Allen relations. International Journal of Geographical Information Science. Medin, D. L., W. D. Wattenmaker, and S. E. Hampson Family resemblance, conceptual cohesiveness, and category construction. Cognitive Psychology 19 (2): Muller, P Topological spatio-temporal reasoning and representation. Computational Intelligence 18 (3): Neisser, U Cognition and Reality: Principles and implications of cognitive psychology. San Francisco, CA: W.H. Freeman. Piaget, J The construction of reality in the child. New York: Basic Books. Pothos, E. M., and N. Chater A simplicity principle in unsupervised human categorization. Cognitive Science 26 (3): Randell, D. A., Z. Cui, and A. G. Cohn A spatial logic based on regions and connections. In Proceedings 3rd International Conference on Knowledge Representation and Reasoning, San Francisco: Morgan Kaufmann. Regier, T The human semantic potential. Spatial language and constraint connectionism. Cambridge (MA), London: The MIT Press.

47 Regier, T., and M. Zheng Attention to endpoints: A cross-linguistic constraint on spatial meaning. Cognitive Science 31 (4): Renz, J Qualitative spatial reasoning with topological information. Berlin: Springer. Riedemann, C Matching names and definitions of topological operators. In Spatial Information Theory. International Conference, COSIT 2005, Elliottville, NY, USA, September 14-18, 2005 ; Proceedings, ed. A. G. Cohn and D. M. Mark, Berlin: Springer. Schwering, A Approaches to semantic similarity measurement for geo-spatial data: A survey. Transactions in GIS 12 (1):2 29. Shariff, A. R., M. J. Egenhofer, and D. M. Mark Natural-language spatial relations between linear and areal objects: The topology and metric of English-language terms. International Journal of Geographical Information Science 12 (3): Siegel, S., and N. J. Castellan Jr Nonparametric statistics for the behavioral sciences. New York: McGraw-Hill. Talmy, L Toward a cognitive semantics. Typology and process in concept structuring. Cambridge, Mass.: MIT Press. Wang, X., P. Matsakis, L. Trick, B. Nonnecke, and M. Veltman A study on how humans describe relative positions of image objects. In Headway in spatial data handling. 13th International Symposium on Spatial Data Handling, ed. A. Ruas and C. Gold, Berlin: Springer. Wang, X., Y. Luo, and Z. Xu SOM: A Novel Model for Defining Topological Line- Region Relations. In Computational Science and Its Applications ICCSA 2004.

48 International Conference Assisi, Italy, May 14-17, 2004 Proceedings, Part III, ed. A. Lagana, M. L. Gavrilova, V. Kumar, Y. Mun, C. J. K. Tan, and O. Gervasi, Berlin: Springer. Weaver, C Building highly-coordinated visualizations in improvise. In Proceedings of the IEEE Symposium on Information Visualization 2004, Austin, TX, October Worboys, M., and M. Duckham Monitoring qualitative spatiotemporal change for geosensor networks. International Journal of Geographical Information Science 20 (10): Worboys, M., M. Duckham, and L. Kulik Commonsense notions of proximity and direction in environmental space. Spatial Cognition and Computation 4 (4): Xu, J Formalizing natural-language spatial relations between linear objects with topological and metric properties. International Journal of Geographical Information Science 21 (4): Yuan, M., and K. S. Hornsby Computation and visualization for the understanding dynamics in geographic domains. A research agenda. Boca Raton, Fla.: CRC Press. Zacks, J. M Using movement and intentions to understand simple events. Cognitive Science 28: Zhan, F. B A fuzzy set model of approximate linguistic terms in descriptions of binary topological relations between simple regions. In Applying soft computing in defining spatial relations, ed. P. Matsakis and L. M. Sztandera, Heidelberg: Physica-Verlag.

49 Figure Captions Figure 1. Depicted is the Rosetta Stone overlaid with a topological conceptual neighborhood graph (CNG) with the topological relations DC (disconnected), EC (externally connected), PO (partial overlap), TPP (tangential proper part), NTPP (nontangential proper part) highlighted (Egenhofer & Al-Taha, 1992; Randell, Cui, & Cohn, 1992). The trajectories of the moving entities in the five scenarios (referred to as semantic domains) are identical from the perspective of topology: they can be characterized by the same path through the conceptual neighborhood graph (DC-EC-PO-TPP-NTPP-TPP-PO- EC-DC). Hence, topology identifies a universal (i.e., invariant) aspect in humans dynamic environments. The critical question is whether topology (and/or other qualitative calculi such as direction calculi) plays the same role, from a cognitive perspective, in each semantic domain. Figure 2. Left: Conceptual neighborhood graph (Egenhofer and Al-Taha 1992; Freksa 1992). Right: Different paths of hurricanes distinguished by ending relations. Depicted are both start and ending relations of the hurricane movement. All hurricanes start in the upper right corner of each icon, disconnected from the peninsula. This setting was held constant across all five different scenarios (Source: Klippel and Li 2009). Figure 3. CatScan user interface (ship scenario). The top half shows the initial screen that a participant sees. The bottom half shows a mockup of an ongoing experiment. In this case, the participant has already created three groups and has started to place icons into these three groups.

50 Figure 4. Boxplots represent the number of groups that participants created in each of the five scenarios. Figure 5. The time participants spent to finalize the grouping task. Figure 6. Similarity matrices visualized with MatrixViewer. The darker the red, the higher the similarities. For the ship matrix, the topological ending relations of the movement are provided. Figure 7. Cluster analysis using Ward s method for all five scenarios. Figure 8. Screenshot of the KlipArt tool. Shown is only the grouping behavior workspace. The figure displays the grouping behavior for two topological equivalence classes: DC1 and EC1 for the cannon scenario (see also Figure 1). Eight icons, numbers , belong to DC1; eight icons, number , belong to EC1. The yellow squares are participants. A participant connected to a group of icons (or an individual icon) means that these icons were grouped together by that participant. Example: Participants 1, 4, 7, 8 10, 14, 20, 21, and 28 placed all 16 icons together into one group. The KlipArt tool allows us to instantly obtain participant details and the linguistic description provided for groups (not depicted). Figure 9. The raw counts of participants who placed all or all but one icon of a topological equivalence class into the same group.

51 Figure 10. Shown is the grouping behavior for EC2 in the ship scenario. Using KlipArt to assess the grouping behavior in detail, it becomes obvious that we have an unusually high count of participants who placed all-but-one icon of this topological equivalence class into the same group (participants: 6, 9, 13, 15, 17, 18, 21). Figure 11. Raw numbers of participants who placed all (or all but one) icon of two topological equivalence classes, which are conceptual neighbors, into the same group. Figure 12. Raw numbers of participants who placed all (or all but one) icons of identical ending relations, into the same group. Figure 13. Two levels of granularity in both Egenhofer s intersection models (Int) and the region connection calculus (compare Knauff, Rauh, and Renz 1997). Figure 14. Shown are (dissimilarity) weights for conceptual neighborhood graphs derived from the cluster analysis shown in Figure 7. The numbers are fusion coefficients derived from Ward s method; they are the conceptual distance at which two clusters (of topologically equivalent relations) are joined together. Figure 15. Zack s model of event segmentation.

52 Figure 1

53 Figure 2

54 Figure 3

55 Figure 4

56 Figure 5

57 Figure 6 Canon Hurricane Geometry DC1 DC2 EC1 EC2 PO1 PO2 NTPP TPP1 TPP2 TPP2 TPP1 NTPP PO2 PO1 EC2 EC1 Ship Tornado DC2 DC1

58 Figure 7 Canon Ship DC1 DC1 EC1 EC1 PO1 PO1 TPP1 TPP1 NTPP NTPP TPP2 PO2 EC2 DC2 Tornado Hurricane Geometry DC1 DC1 DC1 EC1 EC1 EC1 PO1 PO1 PO1 TPP1 TPP1 TPP1 NTPP NTPP NTPP TPP2 TPP2 TPP2 TPP2 PO2 PO2 PO2 PO2 EC2 EC2 EC2 EC2 DC2 DC2 DC2 DC2

59 Figure 8

60 Figure 9

61 Figure 10