Models and hybrid algorithms for inverse minimum spanning tree problem with stochastic edge weights

ISSN 1 746-7233, England, UK World Journal of Modelling and Simulation Vol. 2 (2006) No. 5, pp. 297-311 Models and hybrid algorithms for inverse minimum spanning tree problem with stochastic edge weights Jingyu Zhang 1, Jian Zhou 2 1 School of Sciences Tsinghua University, Beijing 100084, China 2 Department of Industrial Engineering, Tsinghua University, Beijing 100084, China (Received November 8 2005, Accepted January 19 2006) Abstract. An inverse minimum spanning tree problem is to make the least modification on the edge weights such that a predetermined spanning tree is a minimum spanning tree with respect to the new edge weights. In this paper, the inverse minimum spanning tree problem with stochastic edge weights is investigated. The concept of α-minimum spanning tree is initiated, and subsequently an α-minimum spanning tree model and a probability maximization model are presented to formulate the problem according to different decision criteria. In order to solve the two stochastic models, hybrid genetic algorithms and hybrid simulated annealing algorithms are designed and illustrated by some computational experiments. Keywords: minimum spanning tree, inverse optimization, stochastic programming, genetic algorithm, simulated annealing algorithm 1 Introduction Inverse minimum spanning tree problem with stochastic edge weights or stochastic inverse minimum spanning tree (SIMST) problem is an extensive research of inverse minimum spanning tree (IMST). The SIMST problem is a type of inverse optimization problems solving IMST problems whose edge weights are not deterministic numbers but random variables. Inverse optimization problems provide a lot of applications in different regions such as the context of tomographic studies, seismic wave propagation, and in a wide range of statistical inference with prior problems (see [3], [10], [19]). Inverse optimization problems have a common description, i.e., a candidate-feasible solution is given in advance, and the goal is to modify the weight parameters in order to minimize the penalty incurred by the modification of weights. In the IMST problem, a connected and undirected graph is given with weighted edges, and the objective is to modify the edge weights so that a predetermined spanning tree is a minimum spanning tree with respect to new weights, and simultaneously the total modification of weights is a minimum. Much research work has been done for the IMST problem. For instance, Zhang et al. [20] initialized the study of IMST problem and proposed a combinatorial method for calculation. Sokkalingam et al. [18] developed an algorithm with network flow techniques. Then the following research work such as Ahuja and Orlin [2], He et al. [9], Hochaum [10] and Zhang et al. [21] is almost based on the above two original papers. These studies make IMST problem a well developed inverse optimization problem. Simultaneously, the IMST problem is used in a variety of regions, and many applications are appropriately formulated to this problem (see [7]). Actually, in many situations, the parameters in applications are not deterministic. Due to the wide existence of uncertainty, the edge weights may be uncertain variables rather than fixed numbers in some cases. This work was supported by National Natural Science Foundation of China (No. 60425309), and Specialized Research Fund for the Doctoral Program of Higher Education (No. 20020003009). Corresponding author. Tel.: +86-10-62795307; fax: +86-10-62794399. E-mail address: jian@tsinghua.edu.cn. Published by World Academic Press, World Academic Union

298 J. Zhang & J. Zhou: Models and hybrid algorithms Some endeavor has been done to deal with the IMST problem with uncertainty in the literature. For example, to resolve the fuzziness encountered in the IMST problem, Zhang and Zhou [22] investigated a kind of IMST problem with fuzzy edge weights, and some models and algorithms are presented. Although the minimum spanning tree problem with stochastic edge weights has been broadly studied in the literature (see [4], [6], [8], [12], [11]), so far no stochastic programming models and algorithms have been proposed for the IMST problem with stochastic edge weights. In this paper, we will initiate the SIMST problem by some necessary definitions and theorems. Then two stochastic models are provided for the problem, and some hybrid algorithms are developed to solve the models efficiently. This paper is organized as follows. In Section 2, we briefly review the classic IMST problem. Section 3 introduces the SIMST problem including the application backgrounds, the mathematical statement and the definition of α-minimum spanning tree. According to different decision criteria, two stochastic programming models are given in Section 4. In order to solve the proposed models, some hybrid algorithms are suggested and illustrated by some computational experiments in Sections 5 and 6. Finally, some conclusions are drawn in Section 7. 2 Classic inverse minimum spanning tree problem Before the discussion of SIMST problem, it is necessary to review the definition of IMST problem as well as a necessary and sufficient condition of minimum spanning tree and the classic mathematical model for IMST problem. Definition 1 (Minimum Spanning Tree) Given a connected and undirected graph with weighted edges, a minimum spanning tree of the graph is a least-weight tree connecting all vertices. Let us consider a connected and undirected graph G = (V, E) with a predetermined spanning tree T 0, where V is the set of n vertices, and E = {1, 2,, m} is the edge set of G. Let c = (c 1, c 2,, c m ) denote the weight vector, where c i is the weight on edge i. The objective of IMST problem is to find a new edge weight vector x = (x 1, x 2,, x m ) such that T 0 is a minimum spanning tree with respect to x and simultaneously the modification from c to x is a minimum. An example of classic IMST problem with 6 vertices and 9 edges is shown in Fig. 1, where the solid lines represent a predetermined spanning tree T 0. In this case, the goal is to find a new weight vector x = (x 1, x 2,, x 9 ) to minimize 9 i=1 x i c i provided that the predetermined spanning tree T 0 is a minimum spanning tree with respect to x. (c 1, x 1 ) A (c 6, x 6 ) B F (c 2, x 2 ) (c 9, x 9 ) (c 7, x 7 ) (c 5, x 5 ) C (c 8, x 8 ) E (c 3, x 3 ) (c 4, x 4 ) D Fig. 1. An Example of IMST Problem In order to provide a necessary and sufficient condition of minimum spanning tree, some concepts and notations should be introduced in advance. First, we refer to the edges in the predetermined spanning tree T 0 as tree edges, and the edges not in T 0 non-tree edges. For convenience, denote the set of non-tree edges as E\T 0. A path in the graph G is called a tree path if the path contains only tree edges. It is easy to know that WJMS email for contribution: submit@wjms.org.uk

World Journal of Modelling and Simulation, Vol. 2 (2006) No. 5, pp. 297-311 299 there is a unique tree path between the two vertices of any non-tree edge j, called tree path of edge j, whose edge set is denoted by P j. For instance, in Fig. 1, the set of non-tree edges is E\T 0 = {6, 7, 8, 9}, and the tree path of non-tree edge AF is AC-CD-DE-EF, i.e., P 6 = {2, 3, 4, 5}. In Ahuja et al. [1] proved an equivalent condition of minimum spanning tree, called path optimality condition as follows. Theorem 1 (Path Optimality Condition) T 0 is a minimum spanning tree with respect to the edge weight vector x if and only if x i x j 0 for each j E\T 0 and for each i P j (1) where E\T 0 is the set of non-tree edges, and P j is the edge set of tree path of edge j. According to Theorem1, the classic IMST problem can be formulated as the following model, min x m x i c i i=1 (2) subject to : x i x j 0, j E\T 0, i P j where c i and x i are the original and new weights on edge i, i = 1, 2,, m, respectively. Note that the objective function m i=1 x i c i is only one way of measuring total weight modification. The other methods can also be used if necessary (see Sokkalingam et al. [18] ). 3 Stochastic inverse minimum spanning tree problem In this section, the stochastic inverse minimum spanning tree problem is introduced through a traffic investment problem. Then the mathematical description is given and a new concept of α-minimum spanning tree is defined together with an equivalent condition, which will play an important role in modeling the SIMST problem. 3.1 Application backgrounds Many reconstruction problems can be transformed to IMST problems. However, in practice, the weight parameters are not always deterministic due to uncertain environments. In order to see that, let us consider the example of traffic investment problems. Traffic investment problems are commonly used as an infrastructural investment which often cost much money. These traffic networks typically consist of several cities connected by roads or highways. The weights representing road capacities and travel time are always both attached to roads. Obviously, the travel time on roads is usually not fixed numbers but uncertain variables related to road capacities. In many situations, the travel time can be effectively described as a random variable by statistical data. Suppose that there is an old traffic network, in which several cities are interconnected via roads. For some reasons such as traffic congestion, the road capacities must be modified. The decision-maker hopes that a predetermined spanning tree becomes a minimum spanning tree with respect to the travel time on roads in order to ensure high transportation efficiencies on some major roads. Simultaneously, the total modification of road capacities on all roads should be minimized so as to diminish the cost of reconstruction. It is reasonable to assume that the travel time is related to road capacities. Then it is natural to describe the travel time on a road as a random variable whose probability distribution depends on the capacities of roads. This is a typical inverse minimum spanning tree problem with stochastic edge weights, i.e., an SIMST problem. For the SIMST problem, models and algorithms for classic IMST problems become useless. Thus it is necessary to propose some new definitions and models for the problem. 3.2 Mathematical description As a prerequisite of further discussion, notations used are given in advance: WJMS email for subscription: info@wjms.org.uk

300 J. Zhang & J. Zhou: Models and hybrid algorithms G = (V, E): a connected and undirected graph with vertex set V and edge set E = {1, 2,, m}; T 0 : a predetermined spanning tree of G; c i : original weight on edge i, i E; x i : decision variable representing the new weight on edge i, i E; ξ i (x 1, x 2,, x m ): the stochastic weight attached to edge i, whose probability distribution is related to x 1, x 2,, x m, i E. Some concepts are the same as in classic IMST problems, e.g., tree edge, non-tree edge, and tree path of nontree edge. We also denote E\T 0 as the set of non-tree edges, and P j the edge set of tree path of non-tree edge j. The objective of SIMST problem is to find a new edge weight vector x = (x 1, x 2,, x m ) to minimize the modification from weight vector c = (c 1, c 2,, c m ) to x and simultaneously T 0 is a minimum spanning tree with respect to the stochastic edge weight vector ξ(x) = (ξ 1 (x), ξ 2 (x),, ξ m (x)). As an illustration of SIMST problem, a traffic investment problem with 6 cities and 9 roads is showed in Fig. 2, where c i denotes the original edge weight which represents the capacity of road i before reconstruction, x i denotes the new capacity of road i after reconstruction, and ξ i is the stochastic travel time on road i. The solid lines denote the predetermined spanning tree T 0. As a result, there are three different weights attached to each edge including two deterministic ones, c i and x i, respectively representing old and new capacities of road i and a stochastic weight ξ i representing travel time. Note that ξ i = ξ i (x) is a random variable whose probability distribution depends on the new edge weight vector x = (x 1, x 2,, x 9 ). For example, ξ 1 (x) is a normally distributed variable with distribution euler N(x 1 1, 1), whose expected value is x 1 1 and variance is 1. In this case the minimum spanning tree with respect to ξ(x) becomes a stochastic minimum spanning tree which will be well defined in the latter section. (c 1, x 1, ξ 1 ) A (c 6, x 6, ξ 6 ) (c 7, x 7, ξ 7 ) B (c 2, x 2, ξ 2 ) (c 9, x 9, ξ 9 ) F (c 5, x 5, ξ 5 ) C (c 8, x 8, ξ 8 ) E (c 3, x 3, ξ 3 ) (c 4, x 4, ξ 4 ) D Fig. 2. An Example of SIMST Problem 3.3 α-minimum spanning tree In an SIMST problem, the classic definition of minimum spanning tree becomes useless due to the uncertainty of ξ(x). In order to formulate the problem, a new concept of minimum spanning tree with respect to stochastic weights is necessary. Using a given confidence level α, a concept of α-minimum spanning tree is suggested as follows. Definition 2 (α-minimum Spanning Tree) Given a connected and undirected graph with stochastic edge weights and a confidence level α, a spanning tree T 0 is said to be an α-minimum spanning tree if and only if T 0 has a probability not less than α of being a minimum spanning tree with respect to the stochastic weights. From above, we know that the necessary and sufficient condition of minimum spanning tree in Theorem1 provides a useful approach to formulate IMST problem. In random environments, we adapt Theorem1 to obtain a similar result for α-minimum spanning tree as follows. WJMS email for contribution: submit@wjms.org.uk

World Journal of Modelling and Simulation, Vol. 2 (2006) No. 5, pp. 297-311 301 Therom 2 (Stochastic Path Optimality Condition) Given a connected and undirected graph with stochastic edge weight vector ξ and a confidence level α, a spanning tree T 0 is said to be an α-minimum spanning tree if and only if Pr { ξ i ξ j 0, j E\T 0, i P j } α (3) where E\T 0 is the set of non-tree edges, and P j is the edge set of tree path of non-tree edge j. Proof: Suppose that the stochastic weight ξ i on edge i is defined on a probability space (Ω, euler A, P r) for i E, where E = {1, 2,, m} is the edge set of given graph G. Denote a realization of stochastic weight ξ i as ξ i (ω), ω Ω. For each ω Ω, it follows from Theorem 1 that the predetermined spanning tree T 0 is a minimum spanning tree with respect to weight vector ξ(ω) if and only if ξ i (ω) ξ j (ω) 0, j E\T 0, i P j. Definition 2 implies that T 0 is an α-minimum spanning tree if and only if That is, which proves the theorem. Pr { ω Ω T 0 is a minimum spanning tree with respect to ξ(ω) } α. Pr { ω Ω ξ i (ω) ξ j (ω) 0, j E\T 0, i P j } α, 4 Stochastic programming models In this section, an α-minimum spanning tree model is proposed based on the concept of α-minimum spanning tree and the stochastic optimality condition proved above. Moreover, a probability maximization model is also suggested to formulate the SIMST problem according to a different decision criterion. 4.1 α-minimum spanning tree model As a matter of fact, the definition of α-minimum spanning tree and its necessary and sufficient condition have adopted the idea of chance-constrained programming (CCP) initialized by Charnes and Cooper [5] and generalized by Liu [15]. As a type of stochastic programming, stochastic CCP offers a powerful methodology in modeling uncertain decision systems with assumption that stochastic constraints hold with some predetermined confidence levels. In this section, the CCP theory will be used for modeling the SIMST problem based on the concept of α-minimum spanning tree. Recall the traffic investment problem in Section 3. Suppose that investors hope the predetermined spanning tree becomes an α-minimum spanning tree with respect to stochastic travel time on roads to ensure high transportation efficiencies on some major roads, where α is referred to as the confidence level provided as an appropriate safety margin. And the total modification of road capacities should be minimized so as to diminish the reconstruction cost. In this case, a so-called α-minimum spanning tree model can be obtained for SIMST problem according to Definition 2 and Theorem 2 as follows, m min x i c i x i=1 { { Pr ξi (x) ξ j (x) 0, j E\T 0 }, i P j α s.t. l i x i u i, i E (4) where α is the confidence level, l i and u i are lower and upper bounds of x i, respectively. This model means to make the least modification on the deterministic edge weights such that a predetermined spanning tree is an α-minimum spanning tree with respect to stochastic edge weights. WJMS email for subscription: info@wjms.org.uk

302 J. Zhang & J. Zhou: Models and hybrid algorithms 4.2 Probability maximization model Sometimes decision makers expect to ensure that the predetermined spanning tree is a minimum spanning tree with a maximal probability in the SIMST problem. In order to model this type of stochastic decision system, another type of stochastic programming, called dependent-chance programming (DCP), plays a primary role. In the stochastic DCP pioneered by Liu [14], the underlying philosophy is based on selecting the decision with a maximal chance to meet the events. The interested reader is referred to [12] for additional details about DCP theory. When modeling an SIMST problem by DCP theory, a modification supremum S is first provided by the decision-maker. The probability that the modification of edge weights does not exceed S will be maximized with assumption that the predetermined spanning tree T 0 is a minimum spanning tree with respect to stochastic weights. In this way, a probability maximization model is proposed for SIMST problem as follows, { m } max x s.t. Cr i=1 x i c i S { ξi (x) ξ j (x) 0, j E\T 0, i P j l i x i u i, i E where S is the supremum of the total modification of deterministic weights. It seems that the event e characterized by m i=1 x i c i S is a deterministic event with probability either 0 or 1. However, when this event occurs in the random environment ξ i (x) ξ j (x) 0, j E\T 0, i P j, uncertainty is brought to the event e. So, what is the probability of event e in this random environment? In order to explain it more clearly, it is necessary to employ principle of uncertainty stated in Liu [14]. Principle of Uncertainty: The chance function of a random event is the probability that the event is consistent in the uncertain environment. The principle of uncertainty implies that the probability of event e in the uncertain environment ξ i (x) ξ j (x) 0, j E\T 0, i P j is m x i c i S Cr i=1 ξ i (x) ξ j (x) 0, j E\T 0, i P. (6) j If the inequality m i=1 x i c i S holds, the probability in (6) is equivalent to Cr { ξ i (x) ξ j (x) 0, j E\T 0, i P j }. (7) If the inequality is violated, the probability in (6) becomes zero. Thus according to the principle of uncertainty, model (5) can be rewritten as the following programming, max Cr { ξ i (x) ξ j (x) 0, j E\T 0 }, i P j x m x i c i S (8) s.t. i=1 l i x i u i, i E. This model means to maximize the probability of event that a predetermined spanning tree is a minimum spanning tree with respect to stochastic weights under the constraint of a modification supremum. (5) 5 Hybrid algorithms Many algorithms have been developed for solving the classic IMST problem (see [2], [9], [10], [18], [20], [21]). However, they become powerless for solving the stochastic models (4) and (8). In order to solve the above two models, stochastic simulations are designed and then embedded into genetic algorithms and simulated annealing algorithms to produce two different kinds of hybrid algorithms in this section. WJMS email for contribution: submit@wjms.org.uk

World Journal of Modelling and Simulation, Vol. 2 (2006) No. 5, pp. 297-311 303 5.1 Stochastic simulations By uncertain functions we mean the functions with stochastic parameters. In the SIMST models, there is only one type of uncertain functions, i.e., U : x Pr{ξ i (x) ξ j (x) 0, j E\T 0, i P j } (9) which is actually the probability of random event ξ i (x) ξ j (x) 0, j E\T 0, i P j. For some special cases, the probability can be transformed to some crisp equivalents if ξ i (x) ξ j (x) is independent from each other for j E\T 0, i P j. However, the independency is usually not easy to attain. Thus it is more convenient to deal with them by stochastic simulations due to the complexity. Suppose that the edge weight ξ i (x) are random variables defined on a probability space (Ω, euler A, Pr), and denote a realization of ξ i (x) as ξ i (x, ω) for ω Ω, i = 1, 2,, m, respectively. The procedure of stochastic simulations for computing U(x) are shown as follows. Step 1. Set N = 0. Step 2. Generate ω from Ω according to the probability measure Pr. Step 3. If ξ i (x, ω) ξ j (x, ω) 0 for j E\T 0, i P j, then N N + 1. Step 4. Repeat the second and third steps for N times, where N is a sufficiently large number. Step 5. Return U(x) = N /N. 5.2 Hybrid genetic algorithms Genetic algorithms (GAs) have demonstrated considerable success in providing good global solutions to many complex optimization problems. Due to the uncertainty of weight vector ξ(x), stochastic simulations for calculating probability U(x) are embedded into GAs to produce some hybrid genetic algorithms (HGAs) for solving models (4) and (8) efficiently. As an illustration, the following steps show how HGA for model (4) works. Step 1. Initialize pop size feasible chromosomes V k = (x k 1, xk 2,, xk m) for k = 1, 2,, pop size from the potential region [l 1, u 1 ] [l 2, u 2 ] [l m, u m ] uniformly, whose feasibilities are checked by Pr{ξ i (V k ) ξ j (V k ) 0, j E\T 0, i P j } α, k = 1, 2,, pop size, respectively. These probabilities are estimated by stochastic simulations designed above. Step 2. Calculate the objective values O(V k ) = m i=1 xk i c i for all chromosomes V k, k = 1, 2,, pop size, respectively. Rearrange the chromosomes V 1, V 2,, V pop size from good to bad according to their objective values O(V k ) with assumption that a chromosome with smaller objective value is better. Compute the fitness Eval(V k ) of all chromosomes V k, k = 1, 2,, pop size, where Eval(V k ) = a(1 a) k 1, k = 1, 2,, pop size, and a (0, 1) is a parameter in the genetic system. Step 3. Select the chromosomes for a new population by spinning the roulette wheel according to the fitness Eval(V k ) of all chromosomes. First of all, calculate the cumulative probability q k for each chromosome V k, where q 0 = 0 and q k = k j=1 Eval(V j), k = 1, 2,, pop size. Then randomly generate a real number r in (0, q pop size ] and select the ith chromosome V i (1 i pop size) such that q i 1 < r q i. Repeat this process pop size times and obtain pop size chromosomes as a new population. We still denote them by V 1, V 2,, V pop size. Step 4. Update the chromosomes by crossover operation with a predetermined parameter P c, which is called probability of crossover. The procedure of crossover operation is described as follows. (i) Generating a random real number r from the interval [0, 1], the chromosome V k is selected as a parent if r < P c. Repeat this process for k = 1, 2,, pop size and the selected parent chromosomes were denoted by V 1, V 2, (ii) Divide V 1, V 2, into pairs. Then do the crossover operating on each pair according to X = λ V 1 + (1 λ) V 2, Y = (1 λ) V 1 + λ V 2, where λ is a random number generated from the interval (0, 1). WJMS email for subscription: info@wjms.org.uk

304 J. Zhang & J. Zhou: Models and hybrid algorithms (iii) Check the children s feasibilities by stochastic simulations designed in Section 5. If the above two children X and Y are feasible, then replace the two parents with them. Otherwise, we keep the feasible one if it exists, and then redo the crossover operation until two feasible children are obtained. Step 5. Renew the chromosomes by mutation operation with a predetermined probability of mutation P m. Like the process of crossover operation, the mutation operation is performed in the following way. (i) Repeat the following procedure for k = 1, 2,, pop size: generating a random real number r from [0, 1], the chromosome V k is selected as a parent for mutation if r < P m. (ii) Randomly generate a number w from the interval [0, W ] for each selected parent V, where W is an appropriate large number. Then a new chromosome V is obtained by V = V + w d, where d = (d 1, d 2,, d m ) and d i is a random number generated from [ 1, 1]. (iii) Check the feasibility of V by stochastic simulations designed. If V is not feasible, we randomly generate a number w from [0, w] and redo the second step until a new feasible chromosome is obtained, which will replace the selected parent chromosome. Step 6. Repeat the second to fifth steps for N times, where N is a sufficiently large integer. Step 7. Report the best chromosome V as the optimal solution. A similar HGA has been also designed for solving model (8). 5.3 Hybrid simulated annealing algorithms In 1953, Metropolis et al. [17] developed a simulation procedure based on the Monte Carlo method. The method known as a Metropolis Monte Carlo simulation creates a probabilistic acceptance of a Monte Carlo step. This simulation is used to decide whether a new state with energy E n, which is randomly changed from an old state with energy E o, is accepted or rejected. In 1983, Kirkpatrick et al. [13] developed their algorithm after the process of cooling glass from a high temperature to a low one, known as annealing. Metropolis Monte Carlo simulation is well suited for this annealing process. The simulated annealing algorithm (SAA) is therefore an algorithm which simulates the annealing process with Metropolis Monte Carlo simulation as its probabilistic acceptance rule. Metropolis step allows the system to move consistently toward lower energy states, yet still jump out of local minima probabilistically. When the temperature decreases logarithmically, SAA guarantees an optimal solution. For solving the SIMST problem, a hybrid simulated annealing algorithm (HSAA) is suggested. It is constructed based on SAA, and embedded with stochastic simulations. Before describing the algorithm procedure, some notations and parameters are listed as follows. σ o : old state, the candidate solution; σ n : new state, randomly adapted from σ o ; E(σ): energy function of state σ; δ: E(σ n ) E(σ o ) representing improvement in the energy function value; k: Boltzmann constant, which is a small number; α: temperature reduction coefficient; τ 0 : initial temperature; ɛ: terminational temperature, which is usually a number slightly higher than 0. Some HSAAs have been designed to solve the two SIMST models (4) and (8). In order to demonstrate the algorithms clearly, procedures are exhibited as follows for solving the probability maximization model (8). Step 1. Generate the initial state σ o by (x 1, x 2,, x m ) which is randomly selected from the potential region [l 1, u 1 ] [l 2, u 2 ] [l m, u m ]. Redo the above process until σ o is feasible. Then, the state σ o and its energy E(σ o ) become the best-to-date solution and value. The energy function E(σ o ) is chosen as the objective function Pr { ξ i (x) ξ j (x) 0, j E\T 0, i P j }, which is calculated by stochastic simulations designed above. Step 2. Set τ = τ 0 as the initial temperature. Step 3. Repeat the fourth to sixth steps while τ > ɛ. WJMS email for contribution: submit@wjms.org.uk

World Journal of Modelling and Simulation, Vol. 2 (2006) No. 5, pp. 297-311 305 Step 4. Create a new feasible state σ n randomly generated from the neighborhood of σ o, and then compute the improvement of energy δ = E(σ n ) E(σ o ). Step 5. Execute Metropolis step which is listed as follows, (i) If δ 0, set σ o = σ n and E(σ o ) = E(σ n ). If E(σ o ) is higher than the best-to-date value, it will replace the best-to-date value, and simultaneously σ o turns to be the best-to-date solution. (ii) If δ < 0, generate a random number R from (0, 1). If R < e δ/kτ, replace σ o by σ n and E(σ o ) by E(σ n ). Step 6. Cool down the temperature τ α τ. Step 7. Report the best-to-date solution and value. The HSAA for model (4) works in a similar way. 6 Computational experiments In order to illustrate the effectiveness and differences of hybrid algorithms above, some numerical examples are proposed in this section, which are performed on a personal computer. 6.1 Example statement Consider an SIMST problem with 6 vertices and 10 edges showed in Fig. 3, where the vertex set is V = {A, B, C, D, E, F }, the edge set is E = {1, 2,, 10}, and the solid lines represent the predetermined spanning tree, denoted by T 0. There are three weights attached to each edge, where c i and x i represent the original and new deterministic weights of edge i, respectively, and ξ i is the stochastic edge weight. The value of c i and the probability distribution of ξ i are given in Tab. 1. Here the stochastic edge weight ξ i is assumed to be related only to the deterministic edge weight x i of edge i, i.e., ξ i (x) = ξ i (x i ) for the sake of simpleness. (c 1, x 1, ξ 1 ) A B (c 2, x 2, ξ 2 ) (c 3, x 3, ξ 3 ) C (c 8, x 8, ξ 8 ) (c 9, x 9, ξ 9 ) (c 4, x 4, ξ 4 ) (c 5, x 5, ξ 5 ) F (c 6, x 6, ξ 6 ) D (c 7, x 7, ξ 7 ) E (c 10, x 10, ξ 10 ) Fig. 3. SIMST Problem for Computational Experiments 6.2 α-minimum spanning tree model for example In order to minimize the total modification of deterministic edge weights with a given confidence level α = 0.9 so as to diminish the cost of modification, the following 0.9-minimum spanning tree model is obtained, 10 min x i c i x i=1 { { Pr ξi (x i ) ξ j (x j ) 0, j E\T 0 }, i P j 0.9 s.t. 80 x i 180, i E = {1, 2,, 10} where the non-tree edge set E\T 0 = {6, 7, 8, 9, 10}, and P j is the edge set of tree path of edge j E\T 0. Solution process by a hybrid genetic algorithm (10) WJMS email for subscription: info@wjms.org.uk

306 J. Zhang & J. Zhou: Models and hybrid algorithms Table 1. Weights for SIMST Problem in Fig. 3 Edge i Original Weight c i Stochastic Weight ξ i (x i ) 1 100 euler U(x 1 5, x 1 + 5) 2 60 euler U(x 2 10, x 2 + 10) 3 20 euler U(x 3 15, x 3 + 15) 4 160 euler U(x 4 20, x 4 + 20) 5 180 euler U(x 5 25, x 5 + 25) 6 30 euler N(x 6, 5 2 ) 7 50 euler N(x 7, 10 2 ) 8 70 euler N(x 8, 15 2 ) 9 130 euler N(x 9, 20 2 ) 10 150 euler N(x 10, 25 2 ) Performing 4000 cycles in stochastic simulations, the HGA provided in Section 5 is run with 2000 generations. Different groups of parameters in GA system are adopted for comparing, and the result for each group is shown in Tab. 2. The term of Modification in it means the minimum modification of edge weights. From Tab. 2 it appears that all the optimal values differ a little from each other. In order to account for the differentia among them, we present a parameter, called the percent error, i.e., (actual value optimal value)/optimal value 100%, where optimal value is the least one of all the ten minimal modifications obtained. It is named by Error in the last column of Tab. 2. It follows from Tab. 2 that the percent error does not exceed 0.7% when different parameters are used, which implies that HGA is robust to the parameter settings and effective in solving the α-minimum spanning tree model (10). Finally, we choose the least modification Table 2. Solutions of Example by HGA pop size P c P m a Generation Modification Error(%) 1 20 0.3 0.1 0.08 2000 373.43 0.7 2 20 0.2 0.2 0.10 2000 373.28 0.7 3 30 0.1 0.2 0.05 2000 373.21 0.6 4 20 0.1 0.3 0.10 2000 372.85 0.5 5 20 0.1 0.2 0.05 2000 372.41 0.4 6 30 0.3 0.1 0.08 2000 372.20 0.4 7 30 0.1 0.3 0.10 2000 371.99 0.3 8 30 0.3 0.2 0.05 2000 371.76 0.3 9 20 0.3 0.2 0.05 2000 371.47 0.2 10 30 0.2 0.2 0.10 2000 370.82 0.0 in Tab. 2 as the optimal objective value of model (10), i.e., 370.82, whose corresponding optimal solution is x = (93.7324, 80.224, 80.0766, 103.165, 131.615, 101.44, 123.557, 97.8827, 130.104, 156.583). In addition, we illustrate the convergence rate of solution x in Fig. 4. Solution process by a hybrid simulated annealing algorithm Another hybrid algorithm developed for solving model (10), i.e., HSAA, has also been run here. We pick up the parameters: τ 0 = 2 10 8, ɛ = 1 and α = 0.995. Following the procedure illustrated in Section 5, we get an optimal solution x = (95.3755, 80.3473, 80.0374, 102.69, 147.734, 103.522, 122.125, 97.461, 129.583, 173.312) whose corresponding minimal total modification is 371.53. WJMS email for contribution: submit@wjms.org.uk

World Journal of Modelling and Simulation, Vol. 2 (2006) No. 5, pp. 297-311 307 460 440 420 Objective 400 380 0 500 1000 1500 2000 Generations Fig. 4. Illustration of modification with regard to generations in HGA The convergence procedure of this algorithm is illustrated in Fig. 5. Accompanying with the process of temperature decreasing, the objective function gradually approximates the optimal value. However, temperature coordinate is not used in Fig. 5 since the temperature in the algorithm decreases logarithmically. Hence, Iterations indicating the number of times temperature decreased, is used as a substitute for temperature coordinate in the figure. Function value represents the energy in HSAA, representing the coordinate of objective function value. The dashed line signed by Optimum denotes the best-to-date value, and the solid line marked by Objective implies the evolving of the candidate solution E(σ o ). Function value 650 600 550 500 450 400 Objective Optimum 0 2000 4000 6000 Iterations Fig. 5. Illustration of the total modification with regard to iterations in HSAA 6.3 Probability maximization model for example In order to maximize the probability that the total modification of deterministic weights does not exceed 370, we have the following probability maximization model, } { 10 max Pr x i c i 370 x i=1 { ξi (x i ) ξ j (x j ) 0, j E\T 0, i P j s.t. 80 x i 180, i E = {1, 2,, 10} where the non-tree edge set E\T 0 = {6, 7, 8, 9, 10}, and P j is the edge set of tree path of edge j E\T 0. By using the principle of uncertainty, we can translate model (11) to the following programming, max Pr { ξ i (x i ) ξ j (x j ) 0, j E\T 0 }, i P j x 10 x i c i 370 (12) s.t. i=1 80 x i 180, i {1, 2,, 10}. (11) WJMS email for subscription: info@wjms.org.uk

308 J. Zhang & J. Zhou: Models and hybrid algorithms Solution process by a hybrid genetic algorithm A similar procedure of HGA has been run for solving model (12). With 4000 cycles in stochastic simulations and 2000 generations in GA, the results are showed in Tab. 3 with different parameter settings in genetic system. In the table Probability is the maximal probability calculated. In order to measure the differentia among these probabilities, the percent error is proposed and showed in Tab. 3 as Error. It follows from Tab. 3 that the percent error does not exceed 4.1% when different parameters are selected, which implies that the HGA is robust to the parameter settings and effective in solving the probability maximization model (12). Table 3. Solutions of Example by HGA P op size P c P m a Generation Probability Error(%) 1 20 0.3 0.2 0.05 2000 0.8601 4.1 2 20 0.2 0.2 0.10 2000 0.8678 3.3 3 20 0.1 0.3 0.10 2000 0.8683 3.2 4 30 0.1 0.3 0.10 2000 0.8732 2.6 5 20 0.1 0.2 0.05 2000 0.8751 2.5 6 20 0.3 0.1 0.08 2000 0.8765 2.3 7 30 0.2 0.2 0.10 2000 0.8893 0.9 8 30 0.3 0.1 0.08 2000 0.8905 0.7 9 30 0.1 0.2 0.05 2000 0.8968 0.0 10 30 0.3 0.2 0.05 2000 0.8970 0.0 In conclusion, we choose the maximal one among the ten probabilities in Tab. 3 as the optimal objective value, i.e., 0.8970, whose corresponding optimal solution is x = (90.2859, 80.1837, 80.1068, 96.8078, 144.206, 98.6748, 117.003, 96.9749, 129.764, 168.104). An HGA-convergence process of this solution is demonstrated in Fig. 6 as a representative. 0.8 0.6 Objective 0.4 0.2 0 500 1000 1500 2000 Generations Fig. 6. Illustration of modification with regard to generations in HGA Solution process by a hybrid simulated annealing algorithm After selecting parameters, we preset τ 0 = 2 10 8, ɛ = 1 and α = 0.995. Following the procedure proposed in Section 5, we obtain an optimal solution x = (94.2786, 80.3036, 80.3029, 96.7864, 145.352, 103.255, 116.791, 96.0752, 129.901, 169.021) whose corresponding maximal probability is 0.8909. WJMS email for contribution: submit@wjms.org.uk

World Journal of Modelling and Simulation, Vol. 2 (2006) No. 5, pp. 297-311 309 The convergence procedure of this algorithm is illustrated in Fig. 7. In this figure, Iterations represents the number of times temperature decreased, which is a substitute for temperature coordinate. Function value is the energy in HSAA, representing the coordinate of objective function value. The dashed line signed by Optimum denotes the best-to-date value, and the solid line marked by Objective implies the evolving of the candidate solution E(σ o ). 0.8 Optimum Objective Function value 0.6 0.4 0.2 0 0 2000 4000 6000 Iterations Fig. 7. Illustration of the probability with regard to iterations in HSAA 6.4 Algorithm comparison The above numerical examples demonstrate that HGA and HSAA designed in Section5 are both effective for the two SIMST models. Furthermore, it is necessary to discuss the differences between the two types of hybrid algorithms when solving these stochastic models from the following three ways. Quality of solutions Comparing the solutions obtained by HGA and HSAA, it is clear that the optimal solution obtained by HGA is a little better than that by HSAA. It occurs since HSAA may be trapped in local optimum occasionally. However, most of solutions in Tab. 2 and Tab. 3 are worse than those obtained by HSAA. Moreover, bad shapes of the potential regions and the insufficient generations always lead to a cease before convergence when using HGA. If sufficient generations are run, HGA is hopeful to get better solutions. As a result, if accuracy is considered as the most important factor, HGA with sufficient generations is a better choice. From the view of algorithms running time, we may have a different conclusion. The average running time by HSAA is about 270 seconds for α-minimum spanning tree model, and 110 seconds for probability maximization model. However, the running time of HGA is about 9 times of that of HSAA since more chromosomes are demanded in HGA. Thus, when considering the calculation speed, HSAA is recommended, especially for large size problems. Robustness is another important factor that should be considered. In HSAA, the initial temperature and temperature reduction coefficient are not easy to choose. An inappropriate selection of these parameters will cause lack of convergence. Correspondingly, Tab. 2 and Tab. 3 have clearly demonstrated the robustness of HGA to the parameter settings. Different parameter settings will not impact much on the results by HGA when enough generations are run. In a word, for large size problems, HSAA is a better choice with less running time and for small size problems, HGA is more effective for the sake of high quality of solutions and robustness to different parameter settings. 7 Conclusion Inverse optimization problem theory is a subject extensively studied in the context of tomographic studies, seismic wave propagation, and in a wide range of statistical inference with prior problems. As a special WJMS email for subscription: info@wjms.org.uk

310 J. Zhang & J. Zhou: Models and hybrid algorithms type of inverse optimization problem, the inverse minimum spanning tree problem has been discussed detailedly by many researchers. An inverse minimum spanning tree problem is to make the least modification on the edge weights such that a predetermined spanning tree is a minimum spanning tree with respect to the new edge weights. Many reconstruction problems can be transformed to inverse minimum spanning tree problems. Since the weights on edges are often random variables rather than deterministic numbers in practice, it is necessary to investigate the inverse minimum spanning tree problem with stochastic edge weights. Although the minimum spanning tree problem with stochastic edge weights has been broadly studied in the literature, so far no stochastic programming models and algorithms have been proposed for the IMST problem with stochastic edge weights. In this paper, a special type of stochastic inverse minimum spanning tree problem is discussed, i.e., inverse minimum spanning tree problem with stochastic edge weights. In order to formulate this problem, a new concept of α-minimum spanning tree is provided for SIMST problem, and then an equivalent condition of α-minimum spanning tree called stochastic path optimality condition is proved. Based on the concept and stochastic optimality condition, an α-minimum spanning tree model is proposed for SIMST problem. Furthermore, a probability maximization model is also presented to formulate the problem according to a different decision criterion. A variety of classic algorithms have been developed for solving inverse minimum spanning tree problem. However, these algorithms are powerless to solve the two stochastic SIMST models proposed. In order to find the optimal new edge weights satisfying constraints, stochastic simulations for computing probability are designed and then embedded into GA and SA to produce some hybrid algorithms. These hybrid algorithms are illustrated by some numerical examples, whose results show that the algorithms are efficient for solving the stochastic models proposed. References [1] R. K. Ahuja, T. L. Magnanti, J. B. Orlin. Network flows: Theory, algorithms, and applications. New Jersey, 1993. Prentice Hall, Inc. [2] R. K. Ahuja, J. B. Orlin. A faster algorithm for the inverse spanning tree problem. Journal of Algorithms, 2000, 34: 177 193. [3] R. E. Barlow, D. J. Bartholomew, et al. Statistical inference under order restrictions. Wiley, 1972. New York. [4] I. Binmohd. Interval elimination method for stochastic spanning tree problem. Applied Mathematics and Computation, 1994, 66: 325 341. [5] A. Charnes, W. W. Cooper. Chance-constrained programming. Management Science, 1959, 6(1): 73 79. [6] K. Dhamdhere, R. Ravi, M. Singh. On two-stage stochastic minimum spanning trees. Lecture Notes, 2005, 3509: 321C334. [7] A. Farago, A. Szentesi, B. Szviatovszki. Inverse optimization in high-speed networks. Discrete, 2003, 129: 83 98. [8] S. Geetha, K. P. K. Nair. On stochastic spanning tree problem. Networks, 1993, 23: 675 679. [9] Y. He, B. W. Zhang, E. Y. Yao. Weighted inverse minimum spanning tree problems under hamming distance. Journal of Combinatorial Optimization, 2005, 9(1): 91 100. [10] D. S. Hochbaum. Efficient algorithms for the inverse spanning-tree problem. Operations Research, 2003, 51: 785 797. [11] H. Ishii, T. Matsutomi. Confidence regional method of stochastic spanning tree problem. Mathematical and Computer Modelling, 1995, 22: 77 82. [12] H. Ishii, S. Shiode, et al. Stochastic spanning tree problem. Discrete Apllied, 1981, 3: 263 273. [13] S. Kirkpatrick, C. D. Gelatt, M. P. Vecchi. Optimization by simulated annealing. Science, 1983, 220: 671 680. [14] B. Liu. Dependent-chance programming: a class of stochastic optimization. Computers Mathematics with Applications, 1997, 34(12): 89 104. [15] B. Liu. Uncertain programming. John Wiley and Sons, 1999. New York. [12] B. Liu. Theory and Practice of Uncertain Programming. Heidelberg, 2002. [17] N. Metropolis, A. W. Rosenbluth, et al. Equation of state calculations by fast computing machines. Journal of Chemical Physics, 1953, 21: 1087 1092. [18] P. T. Sokkalingam, R. K. Ahuja, J. B. Orlin. Solving inverse spanning tree problems through network flow techniques. Operations Research, 1999, 47: 291 298. WJMS email for contribution: submit@wjms.org.uk

World Journal of Modelling and Simulation, Vol. 2 (2006) No. 5, pp. 297-311 311 [19] A. Tarantola. Inverse problem theory: Methods for data fitting and model parameter estimation. Elsevier, 1987. Amsterdam. [20] J. Zhang, Z. Liu, Z. Ma. On the inverse problem of minimum spanning tree with partition constraints. Mathematical Methods and Operations Research, 1996, 44: 171 187. [21] J. Zhang, S. Xu, Z. Ma. An algorithm for inverse minimum spanning tree problem. Optimization Methods and Software, 1997, 8: 69 84. [22] J. Zhang, J. Zhou. Fuzzy inverse minimum spanning tree. Technical Report, 2005. WJMS email for subscription: info@wjms.org.uk