The evidential reasoning approach for multi-attribute decision analysis under interval uncertainty

Transcription

1 European Journal of Operational Research 174 (2006) Decision Support The evidential reasoning approach for multi-attribute decision analysis under interval uncertainty Dong-Ling Xu, Jian-Bo Yang *, Ying-Ming Wang Manchester Business School, The University of Manchester, PO Box 88, Manchester M60 1QD, UK Received 18 December 2003; accepted 22 February 2005 Available online 14 June Abstract The evidential reasoning (ER) approach is a method for multiple attribute decision analysis (MADA) under uncertainties. It improves the insightfulness and rationality of a decision making process by using a belief decision matrix (BDM) for problem modelling and the Dempster Shafer (D S) theory of evidence for attribute aggregation. The D S theory provides scope and flexibility to deal with interval uncertainties or local ignorance in decision analysis, which is not explored in the original ER approach and will be investigated in this paper. Firstly, interval uncertainty will be defined and modelled in the ER framework. Then, an extended ER algorithm, IER, is derived, which enables the ER approach to deal with interval uncertainty in assessing alternatives on an attribute. It is proved that the original ER algorithm is a special case of the IER algorithm. The latter is demonstrated using numerical examples. Ó 2005 Elsevier B.V. All rights reserved. Keywords: Multiple attribute decision analysis; Evidential reasoning; Interval uncertainty modelling; Intelligent decision system; Decision support systems 1. Introduction For decades, many MADA methods have been developed, such as Analytical Hierarchy Process (AHP) (Saaty, 1988) and Multiple Attribute Utility Theory (Keeney and Raiffa, 1993; Jacquet-Lagreze and Siskos, 1982; Belton and Stewart, 2002). In those methods, MADA problems are modelled using decision matrices, in which an alternative is assessed on each criterion by a single real number. Unfortunately, in many decision situations using a single number to represent a judgement proves to be difficult and sometimes unacceptable. Information would have been lost or distorted in the process of pre-aggregating different types of * Corresponding author. Tel.: ; fax: addresses: l.xu@manchester.ac.uk (D.-L. Xu), jian-bo.yang@manchester.ac.uk (J.-B. Yang) /$ - see front matter Ó 2005 Elsevier B.V. All rights reserved. doi: /j.ejor

2 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) information, such as a subjective judgement, a probability distribution, or an incomplete piece of information, into a single number. It is realised that using distributed assessment instead of single numbers would enable various types of information to be incorporated into a decision making process without pre-aggregation (Zhang et al., 1989) and as a consequence, a belief decision matrix (BDM) concept was conceived. In MADA, performance diversity on higher level attributes in a value tree can help decision makers to gain better insight into alternative decisions in question, which is widely regarded as the primary role of MADA (Phillips, 1982; French, 1995). To take care of the distributed assessment structure contained in a BDM, the D S theory was employed for information aggregation and this resulted in the ER approach (Yang and Singh, 1994; Yang, 2001; Yang and Xu, 2002a). The D S theory was modified in the ER approach in order to provide a rigorous reasoning process for aggregating conflict information, while in its original format the D S theory may generate irrational results if there is conflicting evidence (Yang and Xu, 2002a; Xu and Yang, 2003). The power of the D S theory in handling uncertainties has been explored by many other authors for decision analysis under uncertainties, such as Bauer (1997), Beynon et al. (2000, 2001), Beynon (2002a,b, 2005a,b), Chen (1997), Yager (1992), Yager et al. (1994), Yen (1989). More are listed in Beynon (2002a). In many cases, the D S theory has been used as an alternative approach to Bayes decision theory (Yager, 1992; Yager et al., 1994). Beynon et al. (2000) incorporated the D S theory with the AHP process. The ER approach applies the D S theory within a distributed modelling framework represented by the BDM, and results in a unique family of approaches capable of handling different types of uncertainties including probability uncertainty, ignorance, and fuzziness in various value and preference judgements (Yang and Singh, 1994; Yang and Sen, 1994; Yang and Xu, 2002a,b; Yang, 2001; Yang et al., in press-a, in press-b; Wang et al., 2005b; Liu et al., 2004). From problem modelling point of view, while traditional Bayes theory requires that probabilities can be assigned only to singletons of a set of hypothesises, the D S theory allows belief degrees to be assigned to any subsets, and hence ignorance can be explicitly modelled in a belief matrix. As such, the D S theory is regarded to be more flexible and versatile than the traditional Bayes theory in modelling uncertainty and indeed the latter could be regarded as a special case of the former in knowledge updating given new evidence (Shafer, 1976). More discuss on the potential and advantages of the D S theory in decision making under uncertainty can be found in Beynon et al. (2000). From attribute aggregation point of view, to apply additive utility or value function approaches, all attributes must satisfy the additive or preferential independence condition (Keeney and Raiffa, 1993), which is by no means easy to check, especially when there are more than three attributes in question. However, attribute aggregation using the D S theory only requires the satisfaction of utility or value independence condition (Keeney and Raiffa, 1993), which is much easier to check and is believed to be satisfied in most MADA problems. The D S combination rule can also provide more rigorous yet useful results. For example, it can generate a lower bound and an upper bound of a belief degree to which a hypothesis is believed to be true. However, traditional additive utility or value function approaches do not directly provide such information, though they may also be used to investigate changes caused by imprecise input by conducting sensitivity analysis in a systematic way. In the current ER algorithm, however, decision makers are restricted to provide assessments to individual assessment grades only and any ignorance is assigned to the whole space of grades. Such ignorance is referred to as global ignorance. Experiences show that decision maker may not always be confident enough to provide subjective assessments to individual grades only, but at times wishes to be able to assess beliefs to subsets of adjacent grades. Such ignorance is referred to as local ignorance or interval uncertainty. It is therefore desirable that the ER approach can be enhanced to model the assignment of beliefs to subsets of grades and subsequently process such assessments in decision analysis. Saaty and Vargas (1987) first proposed interval judgments for the AHP method as a way to model subjective uncertainty and used a Monte Carlo simulation approach to find weight intervals from interval

3 1916 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) comparison matrices. Since then, different approaches have been developed to derive priorities from interval comparison matrices (Arbel, 1989; Kress, 1991; Arbel and Vargas, 1993; Salo and Hämäläinen, 1995; Islam et al., 1997; Haines, 1998; Mikhailov, 2002; Wang et al., 2005a,b, in press-a). In this paper, assessments of beliefs to subsets of adjacent grades (or intervals of grades) will be modelled and analysed in the context of MADA. A new ER algorithm, IER, is developed to support the analysis and its properties are explored. The IER algorithm provides a general modelling framework and an attribute aggregation process to deal with both local and global ignorance in MADA. An example is examined to demonstrate the implementation process of the IER algorithm and its applications in decision analysis. Theoretically, belief degrees could be assigned to subsets of grades that are not adjacent and the ER approach could indeed be extended to handle such cases. However, the practical significance of such assignment is not clear and therefore such assignment will not be considered in this paper. In the following sections, interval uncertainty in both quantitative and qualitative attribute assessments are first explained, followed by a brief outline of the original ER algorithm. Then the IER algorithm is developed in details and demonstrated using examples. 2. Multiple attribute assessment with interval uncertainty 2.1. Interval uncertainty under ER assessment framework To begin with, suppose a number of alternatives need to be assessed and (1) without loss of generality, they are assessed using a simple two-level hierarchy of attributes with a father attribute A at the top level and M basic attributes at the bottom level denoted by A ¼fA 1 A 2 A m A n A M g; ð1þ (2) the weights of the attributes are given by x ¼fx 1 x 2 x m x n x M g; ð2þ where x i is the relative weight of the ith basic attribute A i with 0 6 x i 6 1, and P M x i ¼ 1, and (3) N distinctive evaluation grades are defined that collectively provide a complete set of standards for assessing an attribute, as represented by fh 1 H 2 H i H N g. ð3þ Because each individual grade H i can be seen as a grade interval from H i to H i, the N grades can be rewritten as follows: fh 11 H 22 H ii H NN g; ð4þ where H ii is the ith evaluation grade. Without loss of generality, it is assumed that grade H (i+1)(i+1) is preferred to grade H ii. Note that we use A to represent the father attribute, and also the set of its M children attributes. Although the two are conceptually different, they do represent each other. Also note that for the time being it is assumed that all the attributes as given in Eq. (1) use the same set of grades. If not, then different set of grades can always be transformed into a unified set of grades using the rule and utility based grade transformation techniques as discussed by Yang (2001). Based on the above assumptions, an assessment of an alternative on the attribute A m may be mathematically represented as the following distribution: SðA m Þ¼fðH 11 ; b 11;m Þ;...; ðh ii ; b ii;m Þ;...; ðh NN ; b NN;m Þ; ðh 1N ; b 1N;m Þg; ð5þ

4 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) where b ii,m P 0 is the belief degree associated with grade H ii ; P N b ii;m 6 1 and 1 P N b ii;m ¼ b1n;m. b 1N,m represents the unknown portion of the performance that could be assigned to any grades in the whole range denoted by H 1N. However, there may be evidence to indicate that the unknown portion of performance represented by the unassigned belief degree of 1 P N b ii;m may need to be assessed to a narrower range, such as a subset of adjacent grades such as H 13, instead of the whole range H 15 (if N = 5). More generally, the unassigned degree belief could be associated with any grade intervals, denoted by H kl (k =1,..., N, l =1,..., N, k < l). The ER algorithm presented by Yang and Xu (2002a) only deals with the special case with k = 1 and l = N and needs to be extended to deal with the more general interval uncertainties. This is the focus of this paper and we would refer to the extended ER algorithm as Interval ER (IER) algorithm. To better describe the development of the IER algorithm, the ER algorithm of Yang and Xu (2002a) is recapped as follows, with the more generalised notation which helps to verify that the ER algorithm is a special case of the IER algorithm later Outline of the original ER algorithm Suppose A m and A n are two basic attributes assessed using the N grades given in Eq. (4). The assessment of the attribute A m and A n are given in Eqs. (5) and (6) respectively: SðA n Þ¼fðH 11 ; b 11;n Þ;...; ðh ii ; b ii;n Þ;...; ðh NN ; b NN;n Þ; ðh 1N ; b 1N;n Þg. ð6þ Therefore in the current ER approach, each attribute is completely assessed using the following set of grades H: H ¼fH 11 H 22 H ii H NN H 1N g. ð7þ Basic probability mass Suppose x m and x n are the normalised weights for A m and A n. Then the basic probability masses assigned to each element in the set {H} and the unassigned mass in S(A m ) are given by m ii ¼ x m b ii;m ; i ¼ 1;...; N; ð8þ m 1N ¼ x m b 1N;m ¼ x m m H ¼ 1 XN 1 XN m ii þ m 1N ¼ 1 x m b ii;m ; ð9þ b ii;m þ b 1N;m ¼ 1 x m. ð10þ m 1N in Eq. (9) is the probability mass unassigned to individual grades, which is due to the incompleteness (when b 1N 5 0) of the assessment in S(A m ). The m H in Eq. (10) is the remaining probability mass that is unassigned to any evaluation grades or the grade interval H 1N after only A m has been taken into account. In other words, m H represents the remaining role that other attributes can play in the assessment. m H should eventually be assigned to individual grades or H 1N, in a way that is dependent upon the importance of other attributes. Similarly, the basic probability masses assigned to each element in the set {H} and the unassigned mass in the assessment S(A n ) are given by n ii ¼ x n b ii;n ; i ¼ 1;...; N; ð11þ n 1N ¼ x n b 1N;n ¼ x n 1 XN b ii;n ; ð12þ

5 1918 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) n H ¼ 1 XN n ii þ n 1N ¼ 1 x n b ii;n þ b 1N;n ¼ 1 x n. ð13þ The probability masses assigned to each individual grade H ii (i =1,..., N) and the grade interval H 1N, and to be assigned depending on other attributes are all listed in the first row of Table 1 for attributes A m and first column for attribute A n ER algorithm for aggregating two assessments Table 1 is used to illustrate the process of aggregating the two assessments S(A m ) and S(A n ). There are two elements in each cell in the main body of Table 1. The 1st element, such as m ii n jj (i =1,..., N; j =1,..., N), is the combined probability mass jointly assigned by the elements (H ii, b ii,m ) in the assessment S(A m ), or m ii, and (H jj, b jj,n )ins(a n ), or n jj, to the intersection of H ii and H jj. The 2nd element is the intersection of H ii and H jj, which is {H ii }ifi = j or empty set {U} ifi 5 j. Similarly, the elements m 1N n ii is the combined probability masses jointly assigned by m 1N and n ii to the intersection of H 1N and H ii, which is H ii. The same applies to m ii n 1N, m H n ii, m ii n H and m H n H. Note that H is the whole set of individual grades and grade intervals as described in Eq. (7). By aggregating the two assessments, the combined probability mass for each grade {H ii }, denoted by C ii, is generated by adding all the probability mass elements, as shown in Table 1, assigned to {H ii }, which is given by C ii ¼ 1 1 K ½m iin ii þ m ii n 1N þ m 1N n ii þ m ii n H þ m H n ii Š; i ¼ 1;...; N. ð14þ Similarly, the probability mass assigned to the grade interval {H 1N } is given by C 1N ¼ 1 1 K ½m 1Nn 1N þ m 1N n H þ m H n 1N Š ð15þ and the probability mass at large in H ={H 11,..., H NN, H 1N } is given by C H ¼ m Hn H 1 K ; ð16þ where K is the combined probability mass assigned to the empty set {U}: K ¼ XN j¼1 j6¼i m ii n jj ð17þ The scaling factor 1 1 K is used to make sure that P N C ii þ C 1N þ C H ¼ ER algorithm for generating combined belief degrees After aggregation, the combined probability masses given by Eqs. (14) (17) represent the assessment for a new attribute A c which replaces the attributes A m and A n. Therefore the aggregation process in the previous section can be applied recursively to A c and another sibling attribute in the set A as shown in Eq. (1) until the assessments for all the M sibling attributes are aggregated. The recursive aggregation process also applies to the IER algorithm to be discussed later and therefore will not be repeated in the paper. The detailed description of the process can be found in Yang and Xu (2002a). The performances of the M sibling attributes determine the performance of their father attributes A, which is assessed as follows: SðAÞ ¼fðH 11 ; b 11 Þ;...; ðh ii ; b ii Þ;...; ðh NN ; b NN Þ; ðh 1N ; b 1N Þg; ð18þ

6 Table 1 Elements of the combined probability masses for aggregating two assessments A m A n S(A m ) m 11 {H 11 } m 22 {H 22 } m ii {H ii } m (N 1)(N 1) {H (N 1)(N 1) } m NN {H NN } m 1N {H 1N } m H {H} S(A n ) n 11 {H 11 } m 11 n 11 {H 11 } m 22 n 11 {U} m ii n 11 {U} m (N 1)(N 1) n 11 {U} m NN n 11 {U} m 1N n 11 {H 11 } m H n 11 {H 11 } n 22 {H 22 } m 11 n 22 {U} m 22 n 22 {H 22 } m ii n 22 {U} m (N 1)(N 1) n 22 {U} m NN n 22 {U} m 1N n 22 {H 22 } m H n 22 {H 22 } n ii {H ii } m 11 n ii {U} m 22 n ii {U} m ii n ii {H ii } m (N 1)(N 1) n ii {U} m NN n ii {U} m 1N n ii {H ii } m H n ii {H ii } n (N 1)(N 1) {H (N 1)(N 1) } m 11 n (N 1)(N 1) {U} m 22 n (N 1) (N 1) {U} m ii n (N 1) (N 1) {U} m (N 1)(N 1) n (N 1)(N 1) {H (N 1)(N 1) } m NN n (N 1) (N 1) {U} m 1N n (N 1) (N 1) {H (N 1)(N 1) } m H n (N 1)(N 1) {H (N 1)(N 1) } n NN {H NN } m 11 n NN {U} m 22 n NN {U} m ii n NN {U} m (N 1)(N 1) n NN {U} m NN n NN {H NN } m 1N n NN {H NN } m H n NN {H NN } n 1N {H 1N } m 11 n 1N {H 11 } m 22 n 1N {H 22 } m ii n 1N {H ii } m (N 1)(N 1) n 1N m NN n 1N {H NN } m 1N n 1N {H 1N } m H n 1N {H 1N } {H (N 1)(N 1) } n H {H} m 11 n H {H 11 } m 22 n H {H 22 } m ii n H {H ii } m (N 1)(N 1) n H {H (N 1)(N 1) } m NN n H {H NN } m 1N n H {H 1N } m H n H {H} D.-L. Xu et al. / European Journal of Operational Research 174 (2006)

7 1920 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) where b ii (ii =1,..., N)andb 1N are belief degrees with values between 0 and 1 and P N b ii þ b 1N ¼ 1. Suppose the final aggregated probability masses for the M brother attributes are shown as in Eqs. (14) (17). The belief degrees can be calculated from the aggregated probability masses using the following normalisation process to assign C H back to all individual and subsets of grades proportionally (Yang and Xu, 2002a), C ii b ii ¼ ði ¼ 1;...; NÞ; ð19þ 1 C H b 1N ¼ C 1N. ð20þ 1 C H If it is required to rank alternatives, the assessment given in the format of the belief structure as shown in Eq. (18) needs to be transferred into a score. The concept of expected utility or value is used to calculate such a score. Suppose u(h ii ) is the value of the grade H ii with u(h ii+1 )>u(h ii ) if it is assumed that the grade H ii+1 is preferred to H ii. u(h ii ) may be estimated using the probability assignment method (Keeney and Raiffa, 1993) or by constructing regression models using partial rankings or pairwise comparisons (Yang et al., 2001b). If all assessments are complete and precise, there will be b 1N = 0 and the expected value of an alternative on the top attribute A can be used for ranking alternatives, which is calculated by uðaþ ¼ XN b ii uðh ii Þ. An alternative x is preferred to another alternative y on attribute A if and only if u(a(x)) > u(a(y)). If any basic assessment is incomplete, the likelihood to which A may be assessed to H ii is not unique and can be anything in the interval [b ii,(b ii + b 1N )]. In such circumstances, the maximum, minimum and average expected values are calculated as follows: u max ðaþ ¼ XN 1 b ii uðh ii Þþðb NN þ b 1H ÞuðH N Þ; u min ðaþ ¼ðb 11 þ b 1N ÞuðH 11 Þþ XN b ii uðh ii Þ; u avg ðaþ ¼ u maxðaþþu min ðaþ. ð23þ The new IER algorithm The main difference between the ER and IER algorithm is the aggregation process of two attribute assessments. In parallel to the ER algorithm, again suppose A m and A n are two basic attributes assessed using the N grades given in Eq. (3). In the IER algorithm, because the performance of A m or A n can be assessed to an individual grade or a grade interval, the complete set of all individual grades and grade intervals, denoted by H, for assessing each attribute can be represented by 8 9 H 11 H 12 H 1ðN 1Þ H 1N ð21þ ð22þ >< H ¼ H 22 H 2ðN 1Þ H 2N >=. ð24þ >: H ðn 1ÞðN 1Þ H ðn 1ÞN H NN >;

8 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) Note the difference between the sets given by Eq. (7) and Eq. (24). The former is used in the ER algorithm and is a subset of the latter. Based on the above assumption, the assessment of an alternative on attribute A m is then given by SðA m Þ¼fðH ij ; b ij;m Þ; i ¼ 1;...; N; j ¼ i;...; N; i 6 jg; ð25þ and that on the attribute A n is given by SðA n Þ¼fðH ij ; b ij;n Þ; i ¼ 1;...; N; j ¼ i;...; N; i 6 jg ð26þ where b ij,m, b ij,n P 0 is the belief degree associated with the grade interval H ij, and by definition the total belief degrees should be 1, i.e. b ij;m ¼ 1 ð27þ and b ij;n ¼ 1. ð28þ Suppose x m and x n are the normalised weights for A m and A n. Then the basic probability masses assigned to the grade interval {H ij }bys(a m ) are given by m ij ¼ x m b ij;m ði ¼ 1;...; N; j ¼ i;...; NÞ; ð29þ m H ¼ 1 XN x m b ij;m ¼ 1 x m b ij;m ¼ 1 x m. ð30þ Similar to Eq. (10) in the ER algorithm, m H calculated in Eq. (30) is the remaining probability mass that is to be assigned depending on the relative importance of other attributes. m H should eventually be assigned to individual grades and grade intervals in the set H defined by Eq. (24). Similarly, the basic probability masses assigned to the grade interval {H ij }bys(a n ) are given by n ij ¼ x n b ij;n ði ¼ 1;...; N; j ¼ i;...; NÞ; ð31þ n H ¼ 1 XN x n b ij;n ¼ 1 x n. ð32þ To visualise the aggregation process for interval assessment, the above calculated probability masses assigned to each grade interval H ij (i =1,..., N) and to be assigned (in H) depending on other attributes are listed in the first row of Table 2 for the attributes A m and in the first column for the attribute A n. Similar to the ER approach, by aggregating the two assessments, the combined probability mass for each grade interval {H ij } with i 6 j, denoted by C ij, is generated by adding all the probability mass elements, as shown in Table 2, assigned to {H ij }, which is proved in Appendix A and given below: " # C ij ¼ 1 1 K m ijn ij þ Xi ðm kl n ij þ m ij n kl Þþ Xi 1 ðm kj n il þ m il n kj Þþm H n ij þ m ij n H ; ð33þ l¼j and the probability mass at large in H defined by Eq. (24) is given by C H ¼ m Hn H 1 K ; ð34þ

9 1922 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) Table 2 Elements of the combined probability masses of aggregating two assessments

10 where K is the combined probability mass assigned to the empty set {U}: K ¼ XN ðm kl n ij þ m ij n kl Þ. The scaling factor 1 is used to make sure that P N N 1 K P C ij þ C H ¼ 1, which is proved in Theorem 1 in Appendix A. In Eqs. (33) and (35), the summing up process P i 2 i¼i 1 f ðiþ will not be carried out when i 1 > i 2. Mathematically, we may say that X i 2 f ðiþ ¼0 when i 1 > i 2. ð36þ i¼i 1 This convention applies throughout the paper. Similar to the ER algorithm, by applying the above aggregation process recursively until all the M basic attribute assessments are aggregated and assuming that the final resultant probability masses are shown as in Eqs. (33) (35), the overall assessment of A can be expressed as SðAÞ ¼fðH ij ; b ij Þði ¼ 1;...; N; j ¼ i;...; NÞg with b ij ¼ C ij ði ¼ 1;...; N; j ¼ i;...; NÞ. ð37þ 1 C H For ranking alternatives, again expected utilities or values can be calculated. Suppose u(h ii ) is the value of the grade H ii with u(h i+1,i+1 )>u(h ii ) as it is assumed that the grade H i+1,i+1 is preferred to H ii. Because of interval uncertainty, again the maximum, minimum and average expected values are calculated. As the belief degree b ij could be assigned to the best grade in the interval H ij, which is H jj, if the uncertainty turned out to be favourable to the assessed alternative, then the maximum value could be calculated as u max ðaþ ¼ XN b ij uðh jj Þ. Similarly, in the worst case, if the uncertainty turned out to be against the assessed alternative, i.e., the belief degree b ij assigned to H ii, the worst grade in the interval H ij, then the minimum value would be given by u min ðaþ ¼ XN D.-L. Xu et al. / European Journal of Operational Research 174 (2006) b ij uðh ii Þ. The average of the two is given by u avg ðaþ ¼ u maxðaþþu min ðaþ. ð40þ Relationship between ER and IER algorithms In the ER algorithm described in Yang and Xu (2002a), it is assumed that b ij;m ¼ 0 and b ij;n ¼ 0 ðfor all i 6¼ j except i ¼ 1 and j ¼ NÞ. ð41þ If this is the case, then from Eqs. (29) and (31), wehave m ij ¼ 0 ðfor all i < j except i ¼ 1 and j ¼ NÞ; ð42þ n ij ¼ 0 ðfor all i < j except i ¼ 1 and j ¼ NÞ. ð43þ ð35þ ð38þ ð39þ

11 1924 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) Under the above conditions, Eq. (33) becomes C ij ¼ 0 ðfor all i < j except i ¼ 1 and j ¼ NÞ; ð44þ C ii ¼ 1 1 K ½m iin ii þ m ii n 1N þ m 1N n ii þ m ii n H þ m H n ii Š; i ¼ 1;...; N; ð45þ C 1N ¼ 1 1 K ½m 1Nn 1N þ m 1N n H þ m H n 1N Š; ð46þ and K in Eq. (35) becomes K ¼ XN j¼1 j6¼i m ii n jj. ð47þ Eqs. (45), (46), (34) and (47) are exactly the same as Eqs. (14) (17) respectively. That is, the aggregation process of the ER algorithm is a special case of the IER algorithm for b ij,m = 0 (for all m =1,..., M, i 5 j except i = 1 and j = N). 3. Example In this section, two examples will be given. The first one is a simplified problem, a snippet of the second one, and is used to demonstrate the aggregation process of the IER algorithm on a step-by-step basis. The second one is a more realistic problem with a larger scale and is calculated using Excel. For large scale assessment problems, a software package, called Intelligent Decision System, is available and provides user with friendly interfaces for data collection, calculation and result presentation. The examples are based on an article published at the following web site in May 2001: Example 1 for illustration of the IER algorithm Suppose a panel of five judges is commissioned to select talented performers in the entertainment industry. Each performer was assessed on Singing Talent and Instrumental Talent. The scale used to assess each attribute is from 1 to 5 with 1 corresponding to Poor and 5 to Highly Talented. Because Nicole did not perform instruments, she was judged on them using intervals, as shown in Table 3. Table 3 Grades to NicoleÕs sing and instrument performances given by individual judges Attribute (weight) Judge NicoleÕs Grades Singing talent (0.5) Judge 1 4 Judge 2 5 Judge 3 5 Judge 4 5 Judge 5 4 Instrumental talent (0.5) Judge 1 [3 5] Judge 2 [3, 4] Judge 3 [4, 5] Judge 4 [3 5] Judge 5 [4, 5]

12 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) To rank the performance of Nicole with those of the other candidates using the original ER approach, it is simply assumed that the unknown performance grades could be any number between 1 and 5. Based on such an assumption and using Eqs. (21) and (22), the ER algorithm generated a score range for Nicole with the best and worst possible scores being u max (A) = and u min (A) = 0.6 respectively. 1 Note that the uncertainty in her performance score is This result is used to compare the results generated using the IER algorithm, as discussed in the following. The evaluation grades used by the judges are represented as follows: f H 11 H 22 H 33 H 44 H 55 g¼f g. ð48þ Suppose the 5 judges are of equal weights, and as 2 judges gave 4 points and 3 judges gave 5 points to NicoleÕs singing, the distributed assessment of her performance on Singing Talent is given by SðA m Þ¼fðH 44 ; 40%Þ; ðh 55 ; 60%Þg. For simplicity, the weights for Singing and Instrumental Talents are both 0.5, i.e. x m = x n = 0.5 in Eqs. (29) and (31). According to Eqs. (29) and (30), the probability masses assigned to each grade on NicoleÕs Singing Talent are given by m ij ¼ 0 for i ¼ 1;...; 5; j ¼ i;...; 5; except m 44 ¼ ¼ 0.2; m 55 ¼ 0.3 and m H ¼ 1 x m ¼ 0.5. Similarly, the assessment of NicoleÕs Instrumental Talent is given by SðA n Þ¼fðH 35 ; 40%Þ; ðh 45 ; 40%Þ; ðh 34 ; 20%Þg and the probability masses assigned to each grade on NicoleÕs Instrumental Talent are given by n ij ¼ 0 for i ¼ 1;...; 5; j ¼ i;...; 5 except n 35 ¼ 0.2; n 45 ¼ 0.2; n 34 ¼ 0.1; and n H ¼ 0.5. Aggregating the above two assessments using Table 4 or Eqs. (33) (35), we have C ij ¼ 0 ði ¼ 1;...; 5; j ¼ 1;...; 5Þ except K ¼ 0.03; 1 1 K ¼ ; C 44 ¼ 1.03 ð0.04 þ 0.04 þ 0.02 þ 0.1Þ ¼ ¼ 0.21; and similarly C 55 ¼ 0.28; C 34 ¼ 0.05; C 35 ¼ 0.10; C 45 ¼ 0.10; and C H ¼ Suppose the overall performance, after considering Sing Talent and Instrumental Talent together, is given by SðAÞ ¼fðH ij ; b ij Þ; i ¼ 1;...; 5; j ¼ i;...; 5g; SðAÞ ¼fðH 34 ; b 34 Þ; ðh 35 ; b 35 Þ; ðh 44 ; b 44 Þ; ðh 45 ; b 45 Þ; ðh 55 ; b 55 Þg. Then according to Eq. (37) b ij ¼ C ij ði ¼ 1;...; N; j ¼ i;...; NÞ 1 C H ð49þ 1 The calculation is given by the IDS software available from The manual step by step calculation is also available on request from the first author.

13 1926 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) Table 4 Combined probability masses of aggregating assessments on singing talent and instrumental talent

14 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) and 1 C H ¼ ¼ 0.74; we have b ij ¼ 0 for i ¼ 1;...; 5; j ¼ i;...; 5 except b 44 ¼ 0.21=0.74 ¼ 0.28; b 55 ¼ 0.38; b 34 ¼ 0.07; b 35 ¼ 0.14; and b 45 ¼ It is easy to verify that the total sum of the belief degrees is approximately 1 and it should be exactly 1 if there are no rounding errors in Eq. (49) and the subsequent calculations. Suppose the values of the 5 grades shown in Eq. (48) are given by u(h 11 ) = 0.2, u(h 22 ) = 0.4, u(h 33 ) = 0.6, u(h 44 ) = 0.8, and u(h 55 ) = 1, then the expected maximum, minimum and average values, according to (38) (40), are generated as follows: u max ðaþ ¼ XN u min ðaþ ¼ XN b ij uðh jj Þ¼ þ þ þ þ ¼ 0.94; b ij uðh ii Þ¼0.84 and u avg ðaþ ¼ u maxðaþþu min ðaþ 2 ¼ Comparing the above scores with that generated by the ER algorithm in which NicoleÕs Instrumental Talent is assumed to be completely unknown, the expected value interval generated using the ER algorithm is larger ( = 0.333) than that generated using the IER algorithm ( = 0.1). This is because the IER algorithm can incorporate information provided in the interval judgements and therefore reduce ignorance in assessment Example 2: Complete performance assessment The complete picture of the performance assessment problem is discussed as follows. In addition to Nicole, 8 other candidates participated in the performance assessment and were assessed by five judges on the 7 attributes as listed in Table 5. In addition to Singing Talent and Instrumental Talent, the other 5 attributes are Verbal Acting, Non-verbal Acting, Projection Volume, Projection Clarity, and Fun Quotient. The scales used to assess each attribute were the same as used for assessing the Singing and Instrumental Talents, which were from 1 to 5 with 1 corresponding to Poor and 5 to Highly Talented. Table 5 provides the assessment grades given to each performer on each attribute by each judge. Because Jennifer did not perform singing, Nicole, Tiffany and Laura did not perform instruments, the corresponding cells in Table 5 are empty The ER solution To rank the 9 candidates using the ER algorithm, data in Table 5 are transformed into a belief decision matrix as shown in Table 6 except the cells shaded grey. In ER algorithm, it is assumed that the unknown performance grade could be any number from 1 to 5. Therefore the element is each grey cell should be {[1 5], 1)}. Based on this table, the combined assessment results generated using the ER algorithm are given in Tables 7 and 8. Table 7 shows the aggregated assessment results as a distribution for each candidate, while Table 8 shows the maximum, minimum and average scores for each candidate. The average scores are used for ranking purpose. The ranking order is given by Dan 58.24% Laura 52.39% Nicole 59.35% Tiffany 82.61% Jennifer 85.14% David 100% Levi 100% Tim 100% Jonathon ð50þ

15 1928 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) Table 5 Grades to each performer given by individual judge Nicole Tiffany Jennifer Levi Tim Dan Laura Jonathon David Verbal acting (0.15) Judge Judge Judge Judge Judge Non-verbal acting (0.1) Judge Judge Judge Judge Judge Projection volume (0.1) Judge Judge Judge Judge Judge Projection clarity (0.1) Judge Judge Judge Judge Judge Singing talent (0.2) Judge Judge Judge Judge Judge Instrumental talent (0.2) Judge Judge Judge Judge Judge Fun quotient (0.15) Judge Judge Judge Judge Judge where the symbol means more preferable and the percentage above the symbol indicates the extent to which the preference is true. For example, Dan 58.24% Laura means that Dan is more preferable to Laura to an extent of 58.24%. The above ranking is not 100% certain as there are overlaps in the value intervals for the nine performers as shown in Table 8. The percentage is calculated using the equation developed by Wang et al. (in press-a) as outlined below. Suppose there are two interval numbers, [a 1, a 2 ], [b 1, b 2 ] with a 1 6 a 2 and b 1 6 b 2, and a 1, a 2, b 1 and b 2 are all positive. The extent to which the interval number [a 1, a 2 ] P [b 1, b 2 ] is calculated using the following equation: Pða > bþ ¼ maxð0; a 2 b 1 Þ maxð0; a 1 b 2 Þ. ð51þ ða 2 a 1 Þþðb 2 b 1 Þ

16 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) Table 6 Belief matrix of the performance assessment problem

17 1930 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) Table 7 Aggregated performance distribution generated by ER algorithm Degree of Belief Nicole Tiffany Jennifer Levi Tim Dan Laura Jonathon David Grade Grade Grade Grade Grade Grade Table 8 Overall value intervals and ranking generated by ER algorithm Expected value Nicole Tiffany Jennifer Levi Tim Dan Laura Jonathon David Maximum Minimum Average Rank on average value The IER solution Suppose the judges can provide the following grade intervals as shown in Table 9 for the empty cells in Table 5. Then, the distribution assessment for each performer on each attribute is shown in Table 6. The IER algorithm is then used to generate the overall assessments and the ranking of the performers. The aggregated assessments of the performers are as shown in Table 10 and the corresponding value intervals are shown in Table 11. Laura 67.13% Nicole 85.35% Dan 59.93% Tiffany 100% Jennifer 100% David 100% Levi 100% Tim 100% Jonathon ð52þ The ranking of the 9 performers are shown as in Eq. (52). Note that Dan was ranked number 1 by the ER algorithm and now ranked number 3. Again, the ranking is not 100% certain as there are overlaps in the value intervals for the nine performers. However, the ranking shows that Laura, Nicole and Dan are definitely ranked ahead of the other performers. There is no need to gather extra information to separate the Table 9 Interval judgements to each performer given by individual judges Judge Nicole Tiffany Jennifer Laura Singing talent Judge [3, 4] 4 Judge [3 5] 4 Judge [4, 5] 4 Judge [3 5] 4 Judge [3 5] 4 Instrumental talent Judge 1 [3 5] [4, 5] 3 [3 5] Judge 2 [3, 4] [3 5] 3 [3 5] Judge 3 [4, 5] [3 5] 4 [4, 5] Judge 4 [3 5] [3 5] 4 [4, 5] Judge 5 [4, 5] [3 5] 4 [4, 5]

18 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) Table 10 Aggregated performance distribution generated by the IER algorithm Degree of belief Nicole Tiffany Jennifer Levi Tim Dan Laura Jonathon David Grade Grade Grade Grade Grade Grade Grade Grade Table 11 Overall value intervals and ranking generated by the IER algorithm Expected value Nicole Tiffany Jennifer Levi Tim Dan Laura Jonathon David Maximum Minimum Average Average rank former three performers from the other performers. Also, it is more likely or confident that Laura is better than Nicole, who in turn is more likely to be better than Dan. If there is a need to find a winner, it is only necessary to ask Laura and Nicole to perform the instruments; there is no need to examine the singing talent of Jennifer whereby saving costs and avoiding causing frustration for Jennifer. It was described in the originally published article, on which this example is based, that all judges felt that Laura should be the best candidate. However, the original analysis based on traditional multiple attribute value theory simply could not confirm the judgesõ feeling. Using the IER algorithm, Laura is ranked number 1 with a high degree of confidence and the judgesõ feelings are confirmed. 4. Concluding remarks The new development as reported in this paper further extends the capability of the ER approach to utilise information contained in data with local ignorance or interval uncertainty, while in the original framework of the ER approach it is assumed that all ignorance is global. Although it is possible, at least in theory, to find missing information or refine available information, this inevitably results in extra costs and also requires time to conduct such investigation, which may not be a realistic requirement in decision situations where timely decisions have to be made or high cost is involved in finding missing information. On the other hand, decisions could be made without having to gather perfect or complete information as is often done in real life decision making. In cases where more information is needed to support specific decision making such as finding a single winner in a performance assessment problem, the analyses carried out using the new IER approach, as shown in this paper, can provide guidance as to what exact extra information is required to make the decisions thereby avoiding the waste of unnecessary costs and efforts. Traditionally, missing data are dealt with in a number of ways. The simplest way is the so-called list-wise deletion (LD) in statistics, which excludes candidates from further consideration if there are missing data in

19 1932 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) their performance measures. This is perhaps the most widely used approach and is indeed used in some well-known software such as SAS and SPSS. In this way, the best performer can only be chosen from Levi, Tim, Dan, Jonathon and David in our previous example and the gut feeling best, Laura, will be missed out. Another commonly used way of handling missing data is to treat them as zeros or the most unfavourable value. Both ways could introduce distortion to real situations. To minimise distortion from truth, estimated values from different methods are used to replace missing data in many applications, such as in AHP and data envelopment analysis (Harker, 1987; Shen et al., 1992; Carmone et al., 1997; Kao and Liu, 2000; Saen et al., 2005). The mean imputation used in statistical analysis is also one of such methods. Estimated values could be real numbers, intervals, fuzzy numbers, or even distributions, depending on how many people participate in estimation and what methods are used. Since some performance attributes can to certain extent be correlated to each other, correlation and regression analyses as well as neural networks can also be used for missing data estimation, which results in the socalled regression imputation and neural network imputation in statistical analysis. There are other more sophisticated techniques such as multiple imputations (Little and Rubin, 2002; Olinsky et al., 2003; Patrician, 2002). The main difference between the ER approach and other traditional approaches in handling missing data is that the latter either ignore alternatives with missing data or have to use assumed values to replace missing data, while the ER approach accepts missing data in their original formats. In IER, any known or partially known information is used and an unknown value is just assumed to be any values in its possible range. The rationality of the new and original ER algorithms is demonstrated by the performance assessment example and many other applications, such as environmental impact assessment (Wang et al., in press-b), supplier assessment (Teng, 2002; Okundi, 2001; Sonmez et al., 2001; Yang and Xu, 2004), business performance assessment (Siow et al., 2001; Yang et al., 2001a; Xu and Yang, 2003), marine system safety analysis and synthesis (Wang et al., 1995, 1996), software safety synthesis (Wang, 1997; Wang and Yang, 2001), general cargo ship design (Sen and Yang, 1995), retrofit ferry design (Yang and Sen, 1997), product selections (Yang and Singh, 1994), and customer satisfaction survey and result analysis (Yang and Xu, 2003). The use of BDM improves not only the modelling of MADA problems, but also the flexibility in information collection as commented upon by practitioners (company directors, managers and expert assessors), which gives them greater confidence in later stage decision analyses. Although some promising outcomes have been generated, the potential offered by the D S theory in decision analysis under uncertainty and the scope of applying D S theory under various modelling frameworks remain to be explored. Acknowledgement This research was supported by the UK Engineering and Physical Science Research Council under the Grant Nos: GR/N65615/01, GR/R32413/01 and GR/S85498/01. Appendix A. Proof of the interval evidential reasoning algorithm In this section, some conclusions drawn in the paper will be proved. The new evidential reasoning algorithm is based on the Dempster Shafer (D S) theory of evidence and is developed using the set theory and the probability theory. Before we prove the conclusions, the D S evidence combination rule used in the aggregation process of the IER algorithm will be outlined.

20 A.1. DempsterÕs combination rule As indicated by Eqs. (29) and (31), the assessments A m and A n, represented by Eqs. (25) and (26), support an alternative to be assessed to the grade intervals H ij and H kl with the probability masses m ij and n kl. Suppose the intersection of the two grade intervals is H LU. Based on the DempsterÕs combination rules, the IER algorithm dictates that both assessments support the alternative to be assessed to the intersection with a probability mass of m ij multiplied by n kl, i.e., m ij n kl. Table 2 lists all the intersections generated by the pairs of grade intervals H ij (i =1,..., N, j = i,..., N) and H kl (k =1,..., N, l = k,..., N). The corresponding probability mass assigned to each intersection by aggregating the two probability masses m ij and n kl are given in the same cell in the table. When a pair of intervals do not intersect with each other, the intersection is empty and denoted by {U}. If several pairs of intervals generate the same intersection, then the total probability mass of an alternative to be assessed to the intersection is the sum of the aggregated probability masses of those pairs. Eqs. (33) (35) are derived from inspecting Table 2 by identifying the cells with the same intersection H ij. In the next section, with the assistance of the following pictures, it is intended to prove algebraically that Eqs. (33) (35) hold for any N, the number of grades. A.2. Intersection of two grade intervals D.-L. Xu et al. / European Journal of Operational Research 174 (2006) There are three different types of intersections as shown in the following pictures. Fig. 1 shows the inclusion of one set within another set. Fig. 2 shows that one set overlaps but is not included in another set. Fig. 3 shows that one set is completely separated from another set. Note that the basic probability mass assignments (BPA) such as m ij and n ij are marked in the picture for the purpose of indicating whether the grade interval H ij is associated with m ij or n ij. Therefore the magnitudes of those BPA values are not important for this purpose. In Figs. 1 3, it is assumed that intervals H ij, H kl, H il and H kj are associated with n ij, m kl, n il and BPA m kl ( k i ) H kl ( l j ) n ij H ij 0 1 k i j l N Assessment Grade Fig. 1. Grade interval H kl includes grade interval H ij Case I (BPA: basic probability mass assignment). BPA m kj ( k < i ) H kj n il H il ( l >j) 0 1 k i j l N Assessment Grade Fig. 2. Grade interval H kj overlaps with grade interval H il Case I.

21 1934 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) BPA m kl H kl n ij H ij 0 1 k l i j N Assessment Grade Fig. 3. Grade interval H kl is separated from grade interval H ij Case I. m kj respectively Case I. Similar 3 figures can be drawn and are omitted for the case when intervals H ij, H kl, H il and H kj are associated with m ij, n kl, m il and n kj respectively Case II. A.3. IER algorithm To facilitate the proof of the IER algorithm, the following lemma is required. Definition A1. Suppose there are two sets of quadruplets (i, j, k, l), denoted by S 1 and S 2. S 1 is said to be equivalent to S 2, denoted by S 1 () S 2, if any quadruplet (i = i 1, j = j 1, k = k 1, l = l 1 ) belonging to set S 1 also belongs to S 2 and vice versa. Lemma 1. The following sets of quadruplets are equivalent: fði; j; k; lþj1 6 i 6 N; i 6 j 6 N; 1 6 k 6 i; j 6 l 6 Ng m fði; j; k; lþj1 6 k 6 N; k 6 l 6 N; k 6 i 6 l; i 6 j 6 lg; fði; j; k; lþj2 6 i 6 N 1; i 6 j 6 N 1; 1 6 k 6 i 1; j þ 1 6 l 6 Ng m fði; j; k; lþj2 6 i 6 N 1; i þ 1 6 l 6 N; 1 6 k 6 i 1; i 6 j 6 l 1g; fði; j; k; lþj2 6 i 6 N 1; i 6 j 6 N 1; 1 6 k 6 i 1; j þ 1 6 l 6 Ng m fði; j; k; lþj1 6 k 6 N 2; k þ 1 6 j 6 N 1; k þ 1 6 i 6 j; j þ 1 6 l 6 Ng; fði; j; k; lþj2 6 i 6 N; i 6 j 6 N; 1 6 k 6 i 1; k 6 l 6 i 1g m fði; j; k; lþj1 6 k 6 N 1; k 6 l 6 N 1; l þ 1 6 i 6 N; i 6 j 6 Ng. Proof. The proof is trivial because in any of the above four equivalence relations the upper quadruplet set can be deduced from lower set and vice versa. h This lemma provides a foundation that allows the summation process in the IER algorithm to start either with the index i or k, as is required for the proof in the algorithm in Theorem 2. Theorem 1. The combined probability masses generated using the IER algorithm shown in Eqs. (33) (35) have the following relation: C ij þ C H ¼ 1.

22 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) Proof. Let c ij ¼ m ij n ij þ Xi c H ¼ m H n H. ðm kl n ij þ m ij n kl Þþ Xi 1 l¼j ðm kj n il þ m il n kj Þþm H n ij þ m ij n H ; Note that the relation between c ij and C ij, and that between c H and C H are defined as follows: c ij C ij ¼ 1 K ; C H ¼ c H 1 K with K ¼ XN Then we have K ¼ XN ¼ XN c ij ¼ XN ¼ XN þ XN ¼ XN ðm kl n ij þ m ij n kl Þ. " # m ij n ij þ Xi ðm kl n ij þ m ij n kl Þþ Xi 1 ðm kj n il þ m il n kj Þþm H n ij þ m ij n H l¼j " # m ij n ij þ Xi ðm kl n ij þ m ij n kl Þþm ij n H þ m H n ij þ XN 1 1 ¼ XN l¼j ðm kj n il þ m il n kj Þ " # m ij n ij þ Xi ðm kl n ij þ m ij n kl Þþm ij n H þ m H n ij n ij þ XN 1 1 m ij þ Xi l¼j ðm kj n il þ m il n kj Þ l¼j m il n kj þ XN ðm kl n ij þ m ij n kl Þ¼ XN m kl n ij þ XN m kl þ m H þ XN m ij n H ; m ij n kl. X i l¼j ðm kl n ij þ m ij n kl Þ m ij n kl þ XN 1 1 With the help of Lemma 1 and the exchange of the symbols in the summations, we have the following identical equations: m kj n il X i l¼j m ij n kl ¼ XN X l i¼k X l m ij n kl ¼ XN X j k¼i X j m kl n ij ;

23 1936 D.-L. Xu et al. / European Journal of Operational Research 174 (2006) m kj n il ¼ XN 1 l¼iþ1 X l 1 m kj n il ¼ XN 1 þ1 X j 1 l¼i m kl n ij ; 1 1 m il n kj ¼ XN 2 1 X j m il n kj ¼ XN 2 1 X j j¼kþ1 i¼kþ1 þ1 k¼iþ1 m kl n ij ; m ij n kl ¼ XN 1 1 m ij n kl ¼ XN 1 1 i¼lþ1 k¼jþ1 m kl n ij. Therefore, the above P N N P c ij and K can be further expressed as follows: c ij ¼ XN m ij þ Xi m kl þ Xj X j m kl þ m H n ij þ XN 2 1 X j l¼j m kl n ij þ XN þ1 k¼iþ1 k¼i þ XN 1 þ1 m ij n H ; X j 1 l¼i m kl n ij K ¼ XN m kl n ij þ XN 1 1 k¼jþ1 m kl n ij. Note that from the convention given by Eq. (36), ifi = 1 then P i 1 f k ¼ 0; if i = N then P j 1 l¼i f l ¼ 0 for any j 6 N; and if j < i + 1 then P j 1 l¼i f l ¼ 0. Therefore the following equivalence holds: 1 þ1 Similarly, we have 2 1 X j X j 1 l¼i þ1 k¼iþ1 m kl n ij ¼ XN m kl n ij ¼ XN X j X j 1 l¼i m kl n ij. k¼iþ1 m kl n ij ; m kl n ij ¼ XN m kl n ij ; 1 1 k¼jþ1 Therefore, we get m kl n ij ¼ XN k¼jþ1 m kl n ij. c ij ¼ XN n ij þ Xj k¼iþ1 m ij þ Xi m kl þ m H l¼j m kl þ Xj þ XN X j m kl þ Xi 1 k¼i m ij n H ; X j 1 m kl l¼i K ¼ XN n ij m kl þ XN k¼jþ1 m kl.