Cost-based Heuristics and Node Re-Expansions Across the Phase Transition

Transcription

1 Cost-base Heuristics an Noe Re-Expansions Across the Phase Transition Elan Cohen an J. Christopher Beck Department of Mechanical & Inustrial Engineering University of Toronto Toronto, Canaa {ecohen, Abstract Recent work aime at eveloping a eeper unerstaning of suboptimal heuristic search has emonstrate that the use of a cost-base heuristic function in the presence of large operator cost ratio an the ecision to allow re-opening of visite noes can have a significant effect on search effort. In parallel research, phase transitions in problem solubility have prove useful in the stuy of problem ifficulty for many computational problems an have recently been shown to exist in heuristic search problems. In this paper, we show that the impact on search effort associate with a larger operator cost ratio an the number of noe re-expansions is concentrate almost entirely in the phase transition region. Combine with previous work connecting local minima in the search space with such behavior, these observations lea us to hypothesize a relationship between the phase transition an the existence of local minima. 1 Introuction The phase transition in problem solubility has been an important tool in the stuy of problem ifficulty for a number of computational problems (e.g., SAT (Mitchell, Selman, an Levesque 1992; Crawfor an Auton 1996) an CSP (Smith an Dyer 1996; Prosser 1996)). In a recent work, we showe the existence of a rapi transition in problem solubility in heuristic search an the occurrence of the harest problems uring this transition (Cohen an Beck 217). In this paper, we empirically analyze how the phase transition phenomenon interacts with two algorithm esign ecisions: the use of cost-base heuristics on problems with iffering operator cost ratio (Wilt an Ruml 211; 214; Cushing, Benton, an Kambhampati 21; 211) an whether or not to allow re-opening of close noes (Valenzano, Sturtevant, an Schaeffer 214; Sepetnitsky an Felner 215; Sepetnitsky, Felner, an Stern 216). For Greey Best First Search (GBFS) we show the following: 1. The effect of large operator cost ratio on the search effort is most significant in the phase transition region an iminishes outsie. 2. The number of noe re-expansions peaks in the phase transition an eclines outsie. Copyright c 217, Association for the Avancement of Artificial Intelligence ( All rights reserve. 3. exceptionally har problems (Gent an Walsh 1994a; Smith an Grant 1995) appear in the relaxe region of the phase transition where the meian effort is relatively low. 2 The Phase Transition Phenomenon Cheeseman, Kanefsky an Taylor (1991) showe that many NP-complete problems exhibit a transition between regions of solubility an insolubility over a narrow interval of an appropriate problem generation control parameter. One region is uner-constraine with high ensity of solutions an the other is over-constraine with low likelihoo that a solution exists. Furthermore, the harest problems occur in this interval. As note, a significant boy work in the 199s evelope these insights across a number of problem types. Researchers have use the terms mushy region (Smith an Dyer 1996) to refer to the interval an crossover point (Crawfor an Auton 1996) to refer to the point in which the probability that a problem is soluble is.5. We aopt these terms an quantify the mushy region as the range of the control parameter s values in which the observe portion of soluble problems is between.1% an. In a previous work, we showe that both the solubility an problem harness aspects of the phase transition phenomenon can also be observe for GBFS on unit-cost omains (Cohen an Beck 217). As this paper extens this work we present the founational efinitions. Definition 1. (Observe connectivity ensity) Let G V, E be an arbitrary transition graph. We efine the observe connectivity ensity of this graph P(G) = E V ( V 1). Definition 2. (Restricte instance) Let G V, E be an arbitrary transition graph. Ĝ V, Ê is consiere a restricte instance of G if P(Ĝ) < P(G) an Ê E. Definition 3. (Relaxe instance) Let G V, E be an arbitrary transition graph. Ĝ V, Ê is consiere a relaxe version of G if P(Ĝ) > P(G) an Ê E. Moel 1. (p-constraine Benchmark Problems) Given an existing problem s transition graph G V, E an the require connectivity ensity p, the class R G,p consists of all problem instances T, S i, S g such that: 1. T, the transition graph, is a restricte instance of G if p < P(G), or a relaxe instance otherwise. P(T ) = p.

2 2. S i S, a ranomly chosen initial state, k :(S i, S k ) T. 3. S g S, a ranomly chosen goal state such that S g S i an k : (S k, S g ) T. Finally, we use the following control parameter from which the require connectivity ensity p is erive: Expecte number of eges in the transition graph γ := Number of states 3 A Domain-Specific Moel While Moel 1 provies a omain-inepenent way to generate relaxe an restricte problems, for some omains one can fin a omain-specific parameter that controls the constraineness of the problem. For example, in the Gri Navigation omain, the probability of a blocke cell q clearly controls the constraineness of the generate instances: a high q will prouce more constraine state spaces with lower solution ensity while a low q prouces the opposite. 1 We therefore efine a omain-specific moel for Gri Navigation an use it in our analysis. Moel 2. (q-constraine Gri Navigation Problems) Given gri imensions n m, we enote by G nm V, E the transition graph of an n m gri navigation problem an by q the probability of a blocke cell. We efine the class R n,m,q that consists of all problem instances T, S i, S g such that: 1. T, the transition graph, is an instance of G mn in which each cell is blocke with probability q. 2. S i S, a ranomly chosen initial state, k :(S i, S k ) T. 3. S g S, a ranomly chosen goal state such that S g S i an k : (S k, S g ) T. 4 Cost-base Heuristics Definition 4. (Cost-base search; Cushing, Benton, an Kambhampati, 211) A best first search in which g(x) = g c (x), the cost to reach state x, an h(x) = h c (x), an estimation of the cost of the cheapest path from state x to a goal state. Definition 5. (Size-base search; Cushing, Benton, an Kambhampati, 211) A best first search in which g(x) = g (x), the istance (i.e., the number of actions) to reach state x, an h(x) = h (x), an estimation of istance of the shortest path from state x to a goal state. Also calle istance-base search. Definition 6. (Operator cost ratio; Wilt an Ruml, 211) The ratio of the largest ege weight in the graph to the smallest ege weight in the graph. Wilt an Ruml (211) showe, empirically an theoretically, that a larger operator ratio can have a negative effect on the search effort of various best-first search algorithms, incluing GBFS. Their analysis showe that cost-base versions of sliing puzzle an the pancake problem become intractable as the operator cost ratio is increase. The costbase gri navigation problem, however, oes not suffer 1 With q = the state space is still highly constraine, as there are only four actions per state. It is possible to relax the omain further by allowing more actions (e.g., iagonal moves, jumps, etc.). from a significant increase in search effort an the authors attribute it to size-boune local minima an the large number of uplicates. 2 To mitigate the negative effect of large operator cost ratio, a number of authors suggest using sizebase search (Cushing, Benton, an Kambhampati 211; Wilt an Ruml 211; 214). In this section, we present an empirical analysis of sizebase an cost-base search on versions of these three omains plus TopSpin, across the phase transition. For each of the omains, we propose a cost function that is flexible enough to allow us to control the operator cost ratio an examine the change in search effort as we manipulate it. All the experiments in this section use GBFS configure to not re-open close noes an to ranomly break ties in h-values. We generate ranom problem instances for 25 γ values in [, noes 1], with higher ensity insie the phase transition. For each γ value we generate 1 ranom instances. For each instance, we recor its solubility an the number of noes expane in orer to fin a solution or prove that none exists. 4.1 Meian-Case Analysis Gri Navigation. We consier a 5 5 Gri Navigation Problem base on Moel 2. We efine C m (s, a), the cost of applying action a on state s, using a parameter m: 1 m, if a = up 2 m, if a = own C m (s, a) = 3 m, if a = left 4 m, if a = right The parameter m controls the operator cost ratio. As the smallest operator cost is fixe to 1, the operator cost ratio is then 4 m. The heuristic function is base on Manhattan Distance, weighte accoringly. Figure 1 shows the probability of solution an Figure 2 shows the meian number of expane noes, for m {1, 2, 4, 1}, plotte against the probability that a cell is not blocke (1 q). We see a rapi transition in the problem solubility an the peak of the meian search effort is locate in close proximity to the crossover point, as been shown for Moel 1 (Cohen an Beck 217). Even as we increase the operator cost ratio, the meian effort peak remains at the crossover point. To irectly compare the effect of the ifferent cost functions across the phase transition, we analyze the relative number of expane noes. Since the operator cost ratio has no effect on the search effort require to solve insoluble instances (i.e., every noe that is accessible from the initial state will be expane exactly once to prove insolubility), we focus only on the soluble instances. To avoi bias ue to small sample size, we only consier the points in the phase transition in which at least 1% of the problems are soluble. Figure 3 shows the meian ratio of the number of expane noes when using a cost function with higher operator cost ratio (i.e., m {2, 4, 1}) to the search effort when using the lowest operator cost ratio heuristic (). 2 See Fan, Müller, an Holte (217) for recent work.

3 Solubility h Figure 1: 5 5 Gri Navigation: Probability of a solution vs. probability of an unblocke cell Figure 4: 5 5 Gri Navigation: Meian effort ratio between the istance heuristic an a cost heuristic with m=4. # of Expane Noes m = Figure 2: 5 5 Gri Navigation: Meian effort vs. probability of an unblocke cell m = 3 Figure 5: 8-Pancake Problem: Meian effort ratio of soluble instance vs. γ m = Figure 3: 5 5 Gri Navigation: Meian effort ratio of soluble instance vs. probability of an unblocke cell. As Figure 3 clearly shows, the increase in search effort, associate with the large operator ratio, is centere in the region of phase transition. Outsie that region, the effort ratio graually iminishes towars a ratio of one. Figure 4 shows the meian search effort ratio for the 5 5 Gri Navigation Problem (Moel 2) with a large operator ratio (m=4) between the cost heuristic an the istance heuristic. The improvement ue to the istance heuristic is also concentrate in the phase transition region. In fact, the istance heuristic seems to behave similarly to a cost-base heuristic with a lower operator cost ratio (which, of course, it is). Pancake Problem. For the 8-Pancake Problem, base on Moel 1, our cost function is efine base on the bottom pancake in the sub-pile that is about to be flippe. Although somewhat artificial, this cost function is easily incorporate into the gap heuristic (Helmert 21) an allows us to investigate the effect of the operator cost ratio on the search effort. Given z, the size of the lowest pancake in the flippe

4 sub-pile, we efine the cost of the flip to be z m. Again, we use the parameter m to control the operator cost ratio, which is 8 m for the 8-Pancake Problem. Figure 5 shows the meian effort ratio between cost-base search for m {1, 2, 3} an a istance-base search using. The increase in search effort ratio that is associate with larger operator cost ratio is significant only insie the phase transition region an iminishes outsie. The istance heuristic, as before, behaves similarly to a cost-base heuristic with a lower operator cost ratio : 5 : m = : 5 Sliing Tiles. We consier the 3 3 Sliing Tiles Problem base on Moel 1 for which the state space consists of two large isconnecte components (Wilson 1974). In our investigation we generate problems in which the initial state an the goal state are in the same component, an the ae (resp., remove) eges of the relaxe (resp., restricte) instances are within this component. This allows us to observe the full phase transition on one component an to avoi the suen connection of two large components. Wilt an Ruml (211) incorporate costs for the Sliing Tiles problem by assigning ifferent costs for moving each tile. In a later work, they showe that using an inverse cost structure, in which the cost of moving a tile is in inverse correlation to the face of the tile, has a larger expecte local minimum (Wilt an Ruml 214). We therefore use a parametrize version of Wilt an Ruml s cost function in which the cost of moving a tile with a face-value of z is 1 z. The parameter m controls the operator cost ratio, which m is 8 m, for this omain. The heuristic function is base on the stanar Manhattan Distance, weighte by the cost associate with each tile. Figure 6 shows the meian search effort ratio between a cost-base search for m {1, 1.5, 2} an a search base on the istance heuristic. The impact on search effort ratio that is associate with a larger operator cost ratio is concentrate in the phase transition, an iminishes, though is still apparent, as we move away from the phase transition : 5 Figure 6: 3 3 Sliing Tiles: Meian effort ratio of soluble instance vs. γ Figure 7: 1-isk Top: Meian effort ratio of soluble instances vs. γ. TopSpin. Wilt an Ruml (214) foun that in some cases, using a cost-base heuristic requires less search effort than using the istance heuristic that has a lower operator cost ratio, ue to smaller local minima in the cost-base heuristic. We observe such behavior for the TopSpin omain. We consier a 1-isk TopSpin omain with a 4-isk turnstile. Our cost function is base on the sum of faces of the isks in the turnstile: C m (s, a) = ( z) m z T a where T a is the set of faces of the isks in the turnstile, an m is a parameter controlling the operator cost ratio. Figure 7 shows the meian search effort for the TopSpin problem with a PDB heuristic for m {.5, 1, 1.5}, compare to a istance-base heuristic. As expecte, we observe an increase effort as we increase the operator cost ratio. In this case the istance heuristic is not strictly better than the cost-base heuristic for m=.5. This is consistent with Wilt an Ruml s observation. The ifferences iminish as we move away from the phase transition. 4.2 The Harest Instances In the previous section we investigate the effect of operator cost ratio on the meian search effort. Here we examine the harest instances across the constraineness range. Figure 8 shows the -percentile effort ratio for soluble instances of the Gri Navigation Problem (note the log scale on the y-axis for better reaability). The ifferences are significantly larger (i.e., the effort ratio peaks at 4) insie the phase transition region, compare to the meian case. However, high-percentile ratios too, graually iminish towars one as we move away from the phase transition. Interestingly, the peak of the ratio curve slightly shifts to the more relaxe areas, compare to the meian case. Figure 9, Figure 1 an Figure 11 show the - percentile effort ratio for soluble instances of the Pancake Problem, TopSpin an Sliing Tiles Problem respectively. Again, we see that the ratio of the number of expane noes

5 1 1 m = : 5 : m = : Figure 8: 5 5 Gri Navigation: -percentile effort ratio vs. probability of an unblocke cell Figure 1: TopSpin: -percentile effort ratio vs. γ m = : Figure 9: 8-Pancake: -percentile effort ratio vs. γ. insie the phase transition is significantly larger compare to the meian case an it iminishes as we move away from the phase transition. Also, we can see a much stronger shift in peak compare to the meian case. For the Sliing Tiles, we observe that the peak is locate beyon the phase transition region, however as we move away from the peak we see the expecte ecline. The shift in peak suggests that the largest ratio for the harest problems is foun in a more relaxe region of the phase transition. Figure 12 an 13 show the relative effort (compare to the istance heuristic) an absolute effort in the higher percentiles of the Pancake problem with m = 3. We can see that the peak of the highest percentiles of the absolute effort an the peak of the effort ratio both shift to the more relaxe regions in the phase transition. However, the results suggest that this phenomenon is limite to the highest percentiles ( ). Similar results have been observe for the other omains Figure 11: Sliing Tiles: -percentile effort ratio vs. γ % 7% 9% Figure 12: 8-Pancake: -percentile effort ratio vs. γ. 4.3 Discussion The behavior of the meian search effort ratio an the harest instances suggests that the effect of the operator cost ra-

6 # of Expane Noes % 7% 9% # of Noe Re-Expansions % 7% 9% Figure 13: 8-Pancake: -percentile effort vs. γ Figure 14: Gri Navigation: Meian an higher percentiles of noe re-expansions. tio is concentrate insie the phase transition. Our results provie a eeper unerstaning of Wilt an Ruml s observations on the operator cost ratio (Wilt an Ruml 211; 214), empirically emonstrating that they epen on the constraineness of the problem. The anomaly of fining the harest instances in the easy regions of the phase transition has been observe for other types of computational problems (Gent an Walsh 1994a; 1994b; Hogg an Williams 1994). Such problem instances have been terme exceptionally har problems (ehps). The ehps are not simply outliers but rather outliers in an unexpecte region of the phase transition an hence have been the subject of a number of investigations (Gent an Walsh 1994a; Smith an Grant 1995). We previously showe that, for unit-cost problems, the 1%-percentile peaks in the easy region (Cohen an Beck 217). However, that curve is ominate by insoluble instances. This result is not surprising, since heuristic search with a heuristic function that returns a finite value has to exhaust the accessible state space to prove infeasibility. However, the existence of soluble ehps in heuristic search is a new result. The eviation observe for the effort ratio of the harest problems of the Sliing Tiles omain, although small, is an anomaly that may be ue to the interaction of the phase transition phenomenon an the reasons for ehps that requires further stuy to unerstan. 5 Noe Re-Expansions In A*, f(n) = g(n) + h(n) an re-expansions of previously visite noes only occur when using an inconsistent heuristic. In suboptimal search algorithms such as GBFS an Weighte A*, as g(n) is not consiere or is weighte less than h(n), we are not guarantee to avoi re-expansions even when using an amissible an consistent heuristic. Although no theoretical or empirical analysis of which we are aware has specifically aresse re-expansions in GBFS, the authors of several recent works chose to configure GBFS to not re-open close noes, even if a shorter path is foun (Valenzano et al. 214; Xie, Müller, an Holte 214). For Weighte A*, re-expansions can have significant negative effect on the search effort. Valenzano, Sturtevant an Schaeffer (214) presente empirical analysis of reexpansions for Weighte A* on pathfining problems. As they increase the weight on h(n), the proportional search effort spent on re-expansions of visite noes increase. For w = 1, they observe that 91% of the total noe expansions were re-expansions. Sepetnitsky, Felner an Stern (216) performe an empirical analysis for Weighte A* an showe that in more than 9% of the cases, a policy of noe re-opening leas to a search effort that is at least as high as no-reopening, reaching for higher weights. 5.1 GBFS In this section we present an empirical analysis of the effect of noe re-expansions across the constraineness range. All the experiments in this section use GBFS. Naturally, we configure the search to re-open close noes if a cheaper path is foun. As before, we ranomly break h-value ties. We limit the analysis to unit-cost problems an generate 1, ranom problem instances for each of the 25 sample γ values in [, noes 1]. Gri Navigation. Figure 14 shows the meian an higher percentiles of re-expansions across the constraineness range. On average, re-expansions only occur within the phase transition: the meian number of re-expansions outsie of the phase transition region is zero. Even when consiering the higher percentiles, we see that outsie the phase transition the number is much smaller than insie an eclines further from the phase transition region. The reexpansions seem to follow a low-high-low pattern, consistent with the easy-har-easy pattern in search effort. Also, for percentiles, we can see the peak stretches towar areas where the meian is eclining, similar to the ehps. Figure 15 shows the number of re-expansions relative to total expansions (Valenzano, Sturtevant, an Schaeffer 214) an presents a similar pattern.

7 % Noe Re-Expansions % 7% 9% Figure 15: Gri Navigation: Meian an higher percentiles of noe re-expansions. % Noe Re-Expansions % 7% 9% Figure 16: 8-Pancake Problem: Meian an maximal noe re-expansions vs. γ. Pancake Problem. Figure 16 shows the meian an higher percentiles of re-expansions for the 8-Pancake problem. The meian is very low, which can be attribute to the quality of the gap heuristic. However, the trens are clear an similar to the Gri Navigation problem. In the meian case, we see zero re-expansions outsie the phase transition region, an the higher percentiles of re-expansions ecrease as we move away from the phase transition. Again, the peak is wier for the higher percentiles. Similar trens are observe for the relative number of re-expansions. Sliing Tiles. Figure 17 shows the results for the 3 3 Sliing Tiles problem. The re-expansions peak insie the phase transition. The meian reaches zero shortly after leaving the phase transition region. The higher percentiles o not reach zero in the sample γ range, however we can see the ecline as we move away from the phase transition. As expecte, the peak of the higher percentiles is wier for higher percentiles. TopSpin results are similar to those of the Gri Navigation an the Pancake Problem an are omitte ue to space. 5.2 Weighte A* With the interest in noe re-expansions in Weighte A* an the similarity between GBFS an Weighte A* with large weight on h(n), it is interesting to see if the GBFS behavior we have observe can also been seen in Weighte A*. We use the Gri Navigation problem (Moel 2), for which the heuristic function remains amissible an consistent for all values of q. Figures 18 an 19 show the relative an absolute number of expane noes for Weighte A* with ifferent weights on h(n). When w = 1, the number of noe re-expansions is zero everywhere, as expecte. As we increase w, the number of noe re-expansions increases in the phase transition. Consistent with our observation for GBFS, as we move away from the phase transition, the number of re-expansions eclines for all weights to the point it reaches zero. % Noe Re-Expansions % 7% 9% % Noe Re-Expansions w = 8 w = 5 w = 2 w = 1: 5 w = Figure 17: 3 3 Sliing Tiles: Meian an maximal noe re-expansions vs. γ Figure 18: 5 5 Gri Navigation: Meian percent of noe re-expansions vs. probability of an unblocke cell.

8 # of Noe Re-Expansions w = 8 w = 5 w = 2 w = 1: 5 w = Figure 19: 5 5 Gri Navigation: Meian absolute noe re-expansions vs. probability of an unblocke cell. % Noe Re-Expansions w = 8 w = 5 w = 2 w = 1: 5 w = Figure 2: 5 5 Gri Navigation: -percentile of noe re-expansions vs. probability of an unblocke cell. We also analyze the higher percentiles of noe reexpansions for each w value. Figure 2 shows the - percentile of noe re-expansions for w {1, 1.5, 2, 5, 1}. The results show that the -percentile of relative noe re-expansions is also higher as we increase the weight, although it is alreay very high for w = 1.5. Again, the peak of the higher percentiles is wier. 6 Discussion Our results show that the phase transition phenomenon plays an important role in 1) the relation between search effort for cost-base heuristics an operator cost ratio an 2) the relation between search effort an allowing noe re-expansions. Consistent with these observations, we also show that the istance heuristic, that can mitigate the effect of a large operator cost ratio, provies significant improvement only insie the phase transition. It is important to clarify that while the phase transition is useful in preicting the location of the harest problems (both the meian harest an single harest instances), it oes not preict the require search effort. Such a preiction requires taking into account other factors such as the size of the state space an the strength of the heuristic. However, our results show that the phase transition is a key factor in search effort. The Phase Transition an Local Minima. Previous work has shown that the negative effect of increasing the operator cost ratio is ue to the eepening of the local minima, while the istance heuristic tens to have smaller local minima (Wilt an Ruml 211; 214; Cushing, Benton, an Kambhampati 21; 211). Our results show that the effect of increasing the operator ratio, an the benefit often gaine by using the istance heuristic, is significant in the phase transition region, an ecreases as we move away. These results suggest a connection between the constraineness of a problem an the existence of local minima. A reasonable hypothesis is that the likelihoo an/or extent of local minima is much larger in the phase transition an insignificant outsie. A prime area for our future work will be to investigate this hypothesis. The hypothesis is supporte by the iscovery of soluble ehps in heuristic search. Such instances in other types of computational problems are associate with large insoluble subproblems that the search has to exhaust if it enters (Smith an Grant 1997; 1995; Gent an Walsh 1994a). Wilt an Ruml (214) efine local minima in heuristic search as a region that oes not contain the goal but that the search will have to exhaust if it enters. The similarity between these efinitions, as well as their similar location in the phase transition, suggests that they are analogous phenomena. As the large insoluble subproblems are irectly associate with the constraineness of the problem, we conjecture a similar relationship exists for the local minima in satisficing heuristic search. Several methos have been suggeste to mitigate the effect of local minima, incluing the use of ranomization (Valenzano et al. 214) an local exploration (Xie, Müller, an Holte 214; 215). Investigating these methos using the framework of phase transition may yiel interesting new insights. Wilt an Ruml (215) suggeste a quantitative metric to evaluate an compare heuristics for greey best first search, calle the Global Distance Rank Correlation (GDRC), similar to the Fitness-Distance Correlation (FDC) use for other computational problems (e.g., Heckman an Beck, 28). They note that, in general, omains with large local minima have poor GDRC. Investigating the implications of the phase transition to the GDRC/FDC metric is also an interesting open question. 7 Conclusion We performe an empirical analysis of problem instances generate across the phase transition on two aspects of heursitic search algorithms: using cost-base search with varying operator cost ratio an using noe re-expansions. We showe that the effect on search effort associate with a larger operator ratio is concentrate in the phase transition. We also emonstrate that the number of noe re-expansions

9 for both GBFS an Weighte A* peaks in the phase transition an ecreases sharply outsie. Our results suggest that many of the phenomena that are associate with larger search effort are effecte by the phase transition an, therefore, that they shoul be stuie at ifferent level of constraineness. We hypothesize that the existence of local minima in heuristic search problems is closely relate to the phase transition phenomenon. Acknowlegments We thank the anonymous reviewers whose valuable feeback helpe improve the final paper. The authors gratefully acknowlege funing from the Natural Sciences an Engineering Research Council of Canaa. References Cheeseman, P.; Kanefsky, B.; an Taylor, W. M Where the really har problems are. In The Twelfth International Joint Conference on Artificial Intelligence (IJCAI), Cohen, E., an Beck, J. C Problem ifficulty an the phase transition in heuristic search. In The Thirty-First AAAI Conference on Artificial Intelligence (AAAI), Crawfor, J. M., an Auton, L. D Experimental results on the crossover point in ranom 3-SAT. Artificial Intelligence 81(1): Cushing, W.; Benton, J.; an Kambhampati, S. 21. Cost base search consiere harmful. In The Thir Annual Symposium on Combinatorial Search (SoCS), Cushing, W.; Benton, J.; an Kambhampati, S Cost base satisficing search consiere harmful. arxiv preprint arxiv: Fan, G.; Müller, M.; an Holte, R The two-ege nature of iverse action costs. In The Twenty-Seventh International Conference on Automate Planning an Scheuling (ICAPS), in press. Gent, I. P., an Walsh, T. 1994a. Easy problems are sometimes har. Artificial Intelligence 7(1): Gent, I. P., an Walsh, T. 1994b. The harest ranom SAT problems. In KI-94: Avances in Artificial Intelligence, Springer. Heckman, I., an Beck, J. C. 28. Fitness istance correlation an solution-guie constructive search for CSPs. In Proceeings of the Fifth International Conference on Integration of AI an OR Techniques in Constraint Programming for Combinatorial Optimization Problems (CPAIOR8), Helmert, M. 21. Lanmark heuristics for the pancake problem. In Thir Annual Symposium on Combinatorial Search (SoCS), Hogg, T., an Williams, C. P The harest constraint problems: A ouble phase transition. Artificial Intelligence 69(1 2): Mitchell, D.; Selman, B.; an Levesque, H Har an easy istributions of SAT problems. In The Tenth National Conference on Artificial Intelligence (AAAI), Prosser, P An empirical stuy of phase transitions in binary constraint satisfaction problems. Artificial Intelligence 81(1): Sepetnitsky, V., an Felner, A To reopen or not to reopen in the context of Weighte A*. Classifications of ifferent trens. In The Eighth Annual Symposium on Combinatorial Search (SoCS), Sepetnitsky, V.; Felner, A.; an Stern, R Repair policies for not reopening noes in ifferent search settings. In The Ninth Annual Symposium on Combinatorial Search (SoCS), Smith, B. M., an Dyer, M. E Locating the phase transition in binary constraint satisfaction problems. Artificial Intelligence 81(1): Smith, B. M., an Grant, S. A Sparse constraint graphs an exceptionally har problems. In The Fourteenth International Joint Conference on Artificial Intelligence (IJ- CAI), Smith, B. M., an Grant, S. A Moelling exceptionally har constraint satisfaction problems. In International Conference on Principles an Practice of Constraint Programming, Springer. Valenzano, R. A.; Sturtevant, N. R.; Schaeffer, J.; an Xie, F A comparison of knowlege-base GBFS enhancements an knowlege-free exploration. In The Twenty- Fourth International Conference on Automate Planning an Scheuling (ICAPS), Valenzano, R. A.; Sturtevant, N. R.; an Schaeffer, J Worst-case solution quality analysis when not re-expaning noes in best-first search. In The Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI), Wilson, R. M Graph puzzles, homotopy, an the alternating group. Journal of Combinatorial Theory (Series B) 16(1): Wilt, C. M., an Ruml, W Cost-base heuristic search is sensitive to the ratio of operator costs. In The Fourth Annual Symposium on Combinatorial Search (SoCS), Wilt, C. M., an Ruml, W Speey versus greey search. In The Seventh Annual Symposium on Combinatorial Search (SoCS), Wilt, C. M., an Ruml, W Builing a heuristic for greey search. In The Eighth Annual Symposium on Combinatorial Search (SoCS), Xie, F.; Müller, M.; an Holte, R Aing local exploration to greey best-first search in satisficing planning. In The Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI), Xie, F.; Müller, M.; an Holte, R Unerstaning an improving local exploration for GBFS. In The Twenty- Fifth International Conference on Automate Planning an Scheuling (ICAPS),