Risk Assessment of E-Commerce Projects Using Evidential Reasoning

Risk Assessment of E-Commerce Projects Using Evidential Reasoning Rashid Hafeez Khokhar, David A. Bell, Jiwen Guan, and QingXiang Wu The School of Electronics, Electrical Engineering and Computer Science Queen s University Belfast Belfast, BT7 1NN, N.I. UK Tel.: (44)2890974783, (44)2890974165; Fax: (44)2890975666 {r.khokhar, da.bell, j.guan, q.wu}@qub.ac.uk Abstract. The purpose of this study is to develop a decision making system to evaluate the risks in E-Commerce (EC) projects. Competitive software businesses have the critical task of assessing the risk in the software system development life cycle. This can be conducted on the basis of conventional probabilities, but limited appropriate information is available and so a complete set of probabilities is not available. In such problems, where the analysis is highly subjective and related to vague, incomplete, uncertain or inexact information, the Dempster-Shafer (DS) theory of evidence offers a potential advantage. We use a direct way of reasoning in a single step (i.e., extended DS theory) to develop a decision making system to evaluate the risk in EC projects. This consists of five stages 1) establishing knowledge base and setting rule strengths, 2) collecting evidence and data, 3) determining evidence and rule strength to a mass distribution for each rule; i.e., the first half of a single step reasoning process, 4) combining prior mass and different rules; i.e., the second half of the single step reasoning process, 5) finally, evaluating the belief interval for the best support decision of EC project. We test the system by using potential risk factors associated with EC development and the results indicate that the system is promising way of assisting an EC project manager in identifying potential risk factors and the corresponding project risks. Keywords: Dempster-Shafer Theory, Evidential Reasoning, Software Performance Evaluation, Financial Engineering, E-Commerce Application. 1 Introduction Electronic Commerce (EC) is possibly the most promising information technology application that enterprises have seen in recent years. EC addresses the needs of organizations, suppliers and costumers to reduce costs while improving the quality of goods and services, and increasing the speed of service delivery [8]. The current highly competitive business environment needs a good quality EC system, but EC development is subject to various kinds of risk such as malicious code attacks [2], uncertain legal jurisdiction [7], absence of firewall [8], and lack of using cryptography [10]. L. Wang et al. (Eds.): FSKD 2006, LNAI 4223, pp. 621 630, 2006. Springer-Verlag Berlin Heidelberg 2006

622 R.H. Khokhar et al. A comprehensive collection of potential risk factors associated with EC development can be found in Refs [11]. Leung et al., [9] have developed an integrated knowledge-based system that assists project managers to determined potential risk factors and the corresponding project risks. According to this knowledge-base system, most project managers worry about the time involved in risk management when it comes to identifying and assessing risks. However, with the aid of computers and the use of software systems, the time for risk analysis can be significantly reduced. Addison [1] used a Delphi technique to collect the opinion of 32 experts and proposed 28 risks for EC projects. Meanwhile, Carney et al. [3] designed a tool called COTS Usage Risk Evaluation (CURE) to predict the risk areas of COTS products in which he identified four categories comprising 21 risk areas. Ngai and Wat [12] have developed a web-based fuzzy decision support system (FDSS) to assist project manager in identifying potential EC risk factors. However, FDSS has not been tested with real life EC projects, and it can only handle the available risk variables. Also the various membership functions need to be estimated to be as realistically as possible. Cortellessa et al., [4] have introduced a methodology which elaborates annotated UML diagrams to estimate the performance failure probability and combines it with the failure severity estimate which is obtained using the Functional Failure Analysis. This methodology is still have some limitation and only suitable for the analysis of performance-based risk in the early phases of the software life cycle. In this paper, a direct way of reasoning in a single step is presented to develop our evidential reasoning based system to assess the risk in EC projects. A direct way of reasoning in a single step is actually an extended DS theory of evidence. It is a generalization of Yen's [13, 14] model from Bayesian probability theory to the DS theory of evidence. Section 2 presents the system development methodology involved in the system for the risk assessment of EC projects. In section 3, a direct way of reasoning in a single step is described and the experimental results are presented in section 4. Finally, conclusions are given in section 5. 2 System Development Methodology In this paper, a decision making system is developed to assist EC project managers in specifying potential risk factors and evaluating the corresponding EC development risks. Figure 1 presents the methodology involved in the system. Before applying the proposed methodology, it is important to conduct risk identification and compile a list of the most significant uncertainty factors and their descriptions. For this purpose, we use the results of exploratory factor analysis (EFA) by Wat et al., [11] to identifying potential risks associated with EC development. Wat et al., [11] used a source-based approach to categorizing EC development risks is initially used with technical, organizational, and environmental risks as the three primary source categories. Then potential risks associated with EC development was identified with 51 risk items associated with EC development based on a comprehensive literature review and interviewed with EC practitioners.

Risk Assessment of E-Commerce Projects Using Evidential Reasoning 623 The project manager first inputs all risk factor and different pieces of evidence. Then system will search for a rule from the existing rules database and get a rule strength for the selected evidence and hypothesis. The rules database is developed according to the support strengths of different pieces of evidence for different conclusions. The user can edit the rules strengths dynamically, this step will help us to handle uncertain situation. The next step is to combine the outputs from the different rules for all risk factors and its corresponding pieces of evidence using the extended DS engine, and to evaluate the belief intervals for the risk assessment of EC projects. Fig. 1. System development methodology for risk assessment of EC projects 3 The Direct Way of Reasoning in Single Step Guan and Bell [5, 6] extended the DS theory by introducing an evidential mapping that uses mass functions as used in the DS theory to express uncertain relationships. It is a generalization of Yen s [13, 14] model from Bayesian probability theory to the DS theory of evidence. The sections below describe the direct way of reasoning in a single step process through an example of its use. 3.1 Rule Strengths in Expert System In the first stage, we describe the knowledge base and rule strengths. Instead of a mass function being used to express strengths for each element of the evidence space, a mass function is used to express strength for each subset of the evidence space. Consider a frame of discernment with 5 hypotheses Θ = { h1, h3,, h5} : h 1 = " VeryLow", h2 = " Low", h3 = " Medium", = " High", h5 = " VeryHigh". Consider a particular piece of evidence e 1 e.g., wrong schedule estimation, which comes from results of an exploratory factor analysis (EFA) [11] for resource

624 R.H. Khokhar et al. risk factor. We can obtain a general rule which uses this evidence, when present indicated by { e 1 }, strongly supports h = { h1, h3} of Θ and refutes h = { h 4, h }. 5 When the evidence is not present, indicated by { e 1 }, the support strengths are divided between h and Θ. More specifically, we say here that there is an evidence space = e, } and mass functions s s, s : 2 [0,1] such that s s s 1 { 1 e1 11, 12 13 Θ, h3} { e1}) = 0.64, s11({, h5} { e1}) = 0.34, s11 = ({ Θ} { e1}) 0.02;, h3} { e1}) = 0.00, s12 ({, h5} { e1}) = 0.50, s12 = ({ Θ} { e1}) 0.50;, h3} { 1}) = 0.25, s13 ({, h5} { 1}) = 0.45, s13 = ({ Θ} { 1}) 0.30. 11 = 12 = 13 = Guan and Bell [5, 6] used mass function m ( X ) = s( X E) on the power set of hypothesis space Θ to express the rule strength for each subset E of the evidence space. Yen [13, 14] used m ( X ) = s( X e) for each element e of the evidence space to express the rule strength. This means that Guan and Bell [5, 6] have generalized Yen's subset-probability-pair-collection-valued (s-p-p-c-v) mapping to a subset-masspair-collection-valued (s-m-p-c-v) mapping. The s-m-p-c-v mapping Γ from the power set 2 of evidence space to 2 [0,1] 2 Θ 2 [0,1] Γ : (2 { φ }) 2 (1) such that for every non-empty E Γ E) = {( A, s ( A E)),( A, s ( A E)),...,( A, s ( A ))} (2) ( E1 E E1 E2 E E2 En E E Θ E En E Where A E1, AE2,..., AEn 2 ; i. e., A Θ E E1, AE 2,..., AEn are the focal elements of E mass function me ( X ) = se ( X E) on 2 Θ : 0 < se ( AE1 E), se ( AE2 E),..., se ( AEn e E) 1 (3) and Θ (1) A Ei φ for i = 1,...n E ; (2) se ( AEi E) > 0 for i = 1,...n E ; ne (3) = s ( A E) = 1 i 1 E Ei Θ 2 [0,1] Then a rule is a collection RULE =< E, Θ, Γ >, where is an evidence space, Θ is a hypothesis space, and Γ is a s-m-p-c-v mapping from the power set 2 of evidence space to hypothesis space Θ (more precisely, to 2 ). Also, a rule can be expressed by a collection of 2 1 strength mass functions me ( A) = se ( A E) for A Θ, RULE = { se ( A E) E (2 { φ})} (4) is

Risk Assessment of E-Commerce Projects Using Evidential Reasoning 625 2 = { s ( A E ), s ( A E ),..., s ( A E 1 1 2 2 2 1 2 1 { φ } = { E, E,..., E } ; s s, E φ 1 2 2 1 i = Ei i for i = 1,... 2 1. Consider source of evidence e 2 e.g., project over budget which comes from the same results of EFA [11]. This evidence when present indicated by { e 2 }, strongly support subset h = { h3,, h5} of Θ, and refutes h = { h 1, h 2}. When the evidence is not present, indicated by { e 2 }, the support strengths are divided between h and Θ. More specifically, we say here that there is an evidence space 2 = { e2, e2} and mass functions s s, s : 2 Θ [0,1] such that 21, 22 23 )}, s 21 ({ h3,, h5} { e2}) = 0.76, s21({ h1 } { e2}) = 0.20, s21 = ({ Θ} { e2}) = 0.04; s 22 ({ h3,, h5} { e2}) = 0.00, s22 } { e2}) = 0.50, s22 = ({ Θ} { e2}) = 0.50; s 23 ({ h3,, h5} { 2}) = 0.65, s23 } { 2}) = 0.20, s23 = ({ Θ} { 2}) = 0.15. Summarizing, following the method in [5], the knowledge base includes the following rules: RULE-1 IF EVIDENCE { e 1 } THEN h2, h3} WITH STRENGTH s 11, h3} { e1}) = 0. 64 HYPOTHESIS {, h 5} WITH STRENGTH s 11 ({, h5 } e1 ) = 0. 34 WITH STRENGTH s 11 ({ Θ} { e1}) = 0. 02 ELSE IF EVIDENCE { e 1 } THEN h2, h3} WITH STRENGTH s 12, h3} { e1}) = 0. 00 HYPOTHESIS { h 4, h 5} WITH STRENGTH s 12 ({, h5 } { e1 } = 0. 50 WITH STRENGTH s 12 ({ Θ} { e1} = 0. 50 ELSE IF EVIDENCE { 1 } THEN h2, h3} WITH STRENGTH s 13, h3} 1) = 0. 25 HYPOTHESIS {, h 5} WITH STRENGTH s 13 ({, h5 } 1 ) = 0. 45 WITH STRENGTH s 13 ({ Θ} { 1}) = 0. 30 Here 1 = { e1, e1} is an evidence space and m X ) = s ( X ) (5) 11( 11 e1 12 ( X ) s12 ( X e1 ) 13( X ) = s13( X 1 m = (6) m ) (7) are mass functions 2 [0,1] m : 2 [0,1 ] such that m( φ ) = 0, m( Θ) = 1. (8) X Θ

626 R.H. Khokhar et al. RULE-2 IF EVIDENCE { e 2 } THEN HYPOTHESIS { h 3,, h5} WITH STRENGTH s 21 ({ h3,, h5} { e2}) = 0. 76 h 2} WITH STRENGTH s 21 } { e2 } = 0. 20 WITH STRENGTH s 21 ({ Θ} e2 ) = 0. 04 ELSE IF EVIDENCE { e 2 } THEN HYPOTHESIS { h 3,, h5} WITH STRENGTH s 22 ({ h3,, h5} { e2} = 0. 00 h 2} WITH STRENGTH s 22 } { e2 } = 0. 50 WITH STRENGTH s 22 ({ Θ} { e2} = 0. 50 ELSE IF EVIDENCE {Θ} THEN HYPOTHESIS { h 3,, h5} WITH STRENGTH s 23 ({ h3,, h5} { 2} = 0. 65 h 2} WITH STRENGTH s 23 } { 2 }) = 0. 20 WITH STRENGTH s 23 ({ Θ} { 2} = 0. 15 Here 2 = { e2, e12} is an evidence space and m X ) = s ( X ) (9) are mass functions 2 Θ [0,1] 3.2 Data and Evidence 21( 21 e2 m X ) = s ( X ) (10) 22 ( 22 e2 m X ) = s ( X ) (11) 23 ( 23 2 Suppose from the above rules and given a particular confidence in the presence of the data items, we can derive pieces of evidence which are in the conventional DS format. The confidence c 1, we have that e 1 evidence is in fact present is as follows: i.e., we have data strength: _ c 1 ({ e1}) = 0.70, c1 ({ e1}) = 0.20, c1 ( 1) = 0.10 Here c 1 is a mass function over the evidence space 1, intuitively representing the confidence we have that e 1 is present. Similarly, evidence e 2 is present into the following data strengths: _ c 2 ({ e2}) = 0.75, c2 ({ e2}) = 0.20, c2 ( 2 ) = 0.05 Here c 2 is again a mass function over 2. Generally, there is a mass function c i : 2 Θ [0,1] over the evidence space i for i = 1,2,...n. 3.3 Hypothesis Strength In this stage we present the procedure to get from evidence and rule strength to a mass distribution for each rule; i.e., the first half of a single step reasoning process.

Risk Assessment of E-Commerce Projects Using Evidential Reasoning 627 Now, for each rule we can get a hypothesis strength mass function from the evidence strength and the rule strength. For RULE-1; i.e., for rule strengths s 11, s12, s13 and from evidence c 1, the risk variable mass distribution r : 2 Θ [0,1] is obtained as follows. 1 r 1, h3}) = 0.47, r1 ({, h5}) = 0.38, r1 ({ Θ}) = 0.15 By RULE-2; i.e., for rule strengths s 21, s22, s23 and from evidence c 2 we get the following mass distribution r : 2 Θ [0,1] for the other node: 2 r ({ h3,, h5}) = 0.60, r2 }) = 0.26, r2 ({ Θ}) This is the first half of our reasoning process. 2 = 3.4 Combining Prior Mass and Different Rules Now, let us discuss the second half of the single step reasoning process. If μ 1 and μ 2 are two mass functions corresponding to two independent evidential sources, then the combined mass function μ1 μ2 is calculated according to Dempster rule of combination: 1. ( μ 1 μ2 )( φ) = 0 ; 2. For every A Θ, A φ, ( μ1 μ2 )( A) = θ ( Θ, θ φ X Y = A X Y 0.14 μ1( X ) μ2 ( Y ) [ P( A) ] P( X ) P( Y ) μ1( X ) μ2 ( Y ) [ P( θ ) ]) = θ P( X ) P( Y ) For our example, the intersection table of μ1 μ2 for RULE-1 and RULE-2 is shown in the following table 1. Table1. Intersection table to combine two rules (12) μ h, h, }(0.60) h, }(0.26) {Θ}(0.14) 1 μ 2 { 1 2 h3 { 4, h 5 { 3 4 h5 { 1 h 2 h, h, }(0.47) {{ h 3 }(0.20) { h 1 }(0.19) h, h, }(0.07) { 1 2 h3 { h 4, h 5 h }(0.38) h, }(0.36) φ (0) }(0.05) { 4 h 5 { 3,, h5 {Θ}(0.15) h }(0.09) h, }(0.04) Θ (0.02) { 1 h 2 We get the normalization constant (required to discount for mass committed toφ, the empty set) μ1( X ) μ2 ( Y ) N = P( X Y ) = 1.0083 P( X ) P( Y ) X Y φ (13)

628 R.H. Khokhar et al. 3.5 Belief Intervals and Ignorance Finally, we can establish the belief interval for our conclusion after applying this reasoning process. We convert the above results to a set of beliefs for the respective conclusions by adding the masses of all subsets of each conclusion to get the belief allocated to it, and then we get the belief intervals for risk assessment in the EC project using 2 pieces of evidence. So the conclusion is: The resource risk is { h 3 } = Medium using the two pieces of evidence, wrong schedule estimation and project over budget with the belief intervals [ belμ ( A), plsμ ( A)] = [0.2007,0.3657] and ignorance (A) = 0.1657. 4 Case Study The design of the ease of use interface is a key element for the risk assessment of EC developments. Therefore, we design an interface that can be used by any user in EC environments. In this paper, we describe our results with the help of three general steps including: 1) input risk factors and different pieces of evidence 2) edit the rules strengths if required and 3) finally evaluate the belief intervals for the best supported decision. The risk evaluation form is presented in figure 2 to input all potential risk factors, pieces of evidence and data associated with EC project. In this form the project manager/evaluator first inputs all potential risk factors and different pieces of evidence using the results of an EFA [11]. The project manager/evaluator then selects the appropriate hypothesis among five given hypothesis (e.g. VeryLow, Low, Medium, High, and VeryHigh) and assign the data with the help of slider for each evidence. The next step is to determine the rule strengths to get a mass distribution for each rule. Fig. 2. Risk evaluation form to input potential risk associated with EC projects

Risk Assessment of E-Commerce Projects Using Evidential Reasoning 629 The system will search for the rules from the existing rules database for the particular piece of evidence involved. The user can also modify the rule strengths to handle uncertainty. Similarly the user inputs all data/evidence and determines the rules for every risk factor. Finally, we apply the DS engine to evaluate the degrees of belief for these risk factors. The risk assessment results using 10 important risk factors from [11] with belief intervals are presented in table 2. In this table different pieces of evidence (in column 3) are presented for the corresponding risk factors. The next columns demonstrate the assessment results with belief intervals and these risk assessment results are explained in figure 3. For example the first result is described as: The resource risk is Low with support (37%), against (46%) and uncertain (17%). No Table 2. Belief intervals and ignorance for potential risk associated with EC projects Risk factors Variables ( pieces of evidence) Conclusion [ bel μ ( A), plsμ ( A)] ign(a) 1 Resources risk V 21, V 22, V 23, V 24, V 25, V 27 Low [0.3700, 0.5357] 0.1656 2 Requirements risk V 14, V 15, V 16, V 17, V 19, V 20 Very Low [0.1393, 0.2088] 0.0795 3 Vendor quality risk V 46, V 47, V 48, V 49 Very Low [0.0598, 0.1866] 0.1367 4 Client-server security risk V 1, V 2, V 3, V 4, V 5 Low [0.2736, 0.6100] 0.3364 5 Legal risk V 38, V 39, V 40 Very Low [0.0244, 0.0836] 0.0592 6 Managerial risk V 28, V 29, V 30, V 31, V 32 Very Low [0.0815, 0.1111] 0.0296 7 Outsourcing risk V 40, V 41, V 42, V 43, V 45 Very Low [0.3091, 0.7453] 0.4363 8 Physical security risk V 7, V 8, V 9, V 10 Low [0.5284, 0.7513] 0.2229 9 Cultural risk V 50, V 51 Low [0.3958, 0.6727] 0.2769 10 Reengineering risk V 33, V 34 Low [0.0554, 0.2333] 0.1779 Fig. 3. Risk assessment results using direct way of reasoning

630 R.H. Khokhar et al. 5 Conclusions This paper has outlined an approach to the assessment of the risks associated with EC development using a direct way of evidential reasoning with data plus general rule. This method of evidential reasoning has been proposed to assist EC project managers and decision makers in formalizing the types of thinking that are required when assessing the current risk environment of their EC development in a systematic manner. A system has been designed and developed to incorporate is risk analysis model. System evaluation was performed to ascertain whether the system achieved its designed purpose, and the results are satisfactory. The results of the evaluation strongly support the viability of this approach to risk analysis using a direct way of evidential reasoning, and it is demonstrated to be feasible for evaluating EC project risk. References 1. Addison. T.: E-commerce project development risks: evidence from a Delphi survey, International Journal of Information Management. 1 (2003) 25 40. 2. Bandyopadhyay, K., Mykytyn, P, P., Mykytyn, K.: A framework for integrated risk management in information technology. Management Decision, Vol. 37. No. 5. (1999) 437-444. 3. Carney, D., Morris, E., Patrick, R.: Identifying commercial off-the-shelf (COTS) product risks: the COTS usage risk evaluation. Technical Report, CMU/SEI (2003)-TR-023. 4. Cortellessa, V., Goseva-Popstojanova, K., Kalaivani Appukkutty, Guedem, A.R., Hassan, A., Elnaggar, R., Abdelmoez, W., Ammar, H.H., Model-based performance risk analysis Software Engineering, IEEE Transactions on Vol. 31. Issue 1. (2005) 3-20 5. Guan, J. W., Bell, D. A.: Evidence theory and its applications. Vol.1, Studies in Computer Science and Artificial Intelligence 7, Elsevier, The Netherlands, (1991). 6. Guan, J. W., Bell, D. A.: Evidence theory and its applications. Vol.2, Studies in Computer Science and Artificial Intelligence 8, Elsevier, The Netherlands, (1992). 7. Halpern, J., Mehrotra, A. K.: The tangled web of e-commerce: identifying the legal risks of online marketing. The computer Lawyer, Vol. 17, No. 2, 8-14. 8. Kalakota, R., Whinston, A.B.: Frontiers of the Electronic Commerce., Addison-Wesley, Reading, MA (1996). 9. Leung, H.M., Chuah, K.B., Tummala, V.M.R.: A knowledge-based system for identifying potential project risks. OMEGA: International Journal of Management Science. Vol. 26, Issue. 5. (1998) 623 638. 10. Treese, G. W., Stewart, L. C.: Designing Systems for Internet Commerce, Addison Wesley, Massachusetts. 11. Wat, F. K. T., Ngai, E.W.T., Cheng, T.C.E.: Potential risks to e-commerce development using exploratory factor analysis. International Journal of Services Technology and Management (2004), Vol. 6. Part 1, Pages 55-71. 12. Ngai, E.W.T., Wat, F.K.T.: Fuzzy decision support system for risk analysis in e-commerce development. Journal of Decision Support Systems. Vol. 40. Issue 2. (2005) 235-255 13. Yen, J.: A reasoning model based on an extended Dempster-Shafer theory. Proceedings AAAI-(1986) 125-131. 14. Yen, J.: GERTIS: A Dempster-Shafer Approach to Diagnosing Hierarchical Hypotheses", Communications of the ACM 5 Vol. 32, (1989), 573-585.