Identification of Data Sets for a Robustness Analysis. RDDC Valcartier, 2 Université Laval 10 th ICCRTS

Transcription

1

2 Identification of Data Sets for a Robustness Analysis Micheline Bélanger 1, Jean-Marc Martel 2, Adel Guitouni 1 1 RDDC Valcartier, 2 Université Laval 10 th ICCRTS Defence Research and Development Canada Recherche et développement pour la défense Canada Canada

3 Agenda Canadian Forces Operations Planning Process (CFOPP) Multicriteria Decision Aid (MCDA) Methodology Robustness Analysis Identification of possible data sets based on DM local preferences Identification of plausible data sets

4 Canadian OPP and Estimate Process COAs Development COAs Refinement COAs War gaming COAs Comparison Information Brief Decision Brief Political & Military Assessment Initiating Directive Initiation Orientation Concept Development Plan Development Plan Review CPG Mission Analysis CONOPs OPLAN Mission Statement Estimate Process SOR

5 Comparison of COAs Decision Maker s Preferences COA1 COA2... COAm COA2 COAm... COA1 Set of COAs Ranking or Selection

6 Initial Decision Maker s Preference: COAs Evaluation Criteria Covering Operational Tasks Factors Flexibility Covering Enemy s CoAs Covering Mission s Locations Criteria Personnel Effectiveness Impact of Sensors Coverage Gaps Risk Military Personnel Loss Collateral Damages Confrontation Risk Equipment Reliability Operation Complexity Complexity Logistic Complexity C&C Complexity Sustainability Optimum Use of Resources Sustainability Cost of Resources

7 Comparison of COAs COA1 COA2... COAm Best possible compromise Decision Maker s considering: Conflicting Preferences evaluation criteria DM s values and preferences COA2 COAm... COA1 Set of COAs Ranking or Selection

8 Comparison of COAs COA1 COA2... COAm Decision Maker s Preferences MCDA Methodology COA2 COAm... COA1 Set of COAs Ranking or Selection

9 Applying MCDA Methodology to Compare COAs Criteria (1 n) CoAs (1 m) C 1... C... C n a 1 e e 1... e 1n : : : : : : a i e i1... e i... e in : : : : : : a m e m1... e m... e mn DM s Preferences Multicriterion Aggregation Procedure CoA1 CoA2... CoAm Local Preferences

10 Modelling Decision Making Styles: Introduction of Local Preferences Modelling Each criterion is assigned a coefficient of relative importance (π ), which might represents a trade-off or a voting power When comparing two COAs, three types of thresholds are introduced Indifference (q ) thresholds Preference (p ) thresholds Veto (v ) thresholds

11 Modelling Decision Making Styles: Introduction of Local Preferences Modelling (2) Indifference (q ) thresholds represent: the highest difference between the evaluations of two COAs, according to a given criterion, for which the decision-maker is incapable to make a clear choice between these two alternatives, given that everything is the same otherwise Preference (p ) thresholds represent: the smallest difference between the evaluations of two alternatives, according to a given criterion, for which the decision-maker is able to make a clear choice of one, given that everything is the same otherwise Veto (v ) thresholds represent: the smallest difference between the evaluations of two alternatives, according to a given criterion, for which the decision-maker cannot conclude that an alternative a i is as good as a k, if the performance of a k is higher than the performance of a i and if the difference of the evaluations between them is greater than ν (even if the performance of a i is higher than a k for all others criteria).

12 strict preference of a i over a k Possible Preference Relationship When Comparing two COAs weak preference of a i over a k indifference between a i and a k weak preference of a k over a i strict preference of a k over a i a i φ a k a i φ f a k a i ~ a k a kφ f ai ak φ a i e p i e q i ei e + i q e + i p e k Discrimination Thresholds ai φ ak ei e f ai φ ak q < e ai ~ ak ei e where p > q k i k + p e q k < p

13 Problematic Modelling requires transforming, reduction and decomposition of the reality It is impossible to derive exact models of the situation The complexity of the military operation context prevents from deriving exact and precise values to represent Commanders preferences structures (command style) Very high likelihood for more than one plausible data set to represent the Decision Maker s preferences structure Possibility to get more that one optimal solution for the same decision-making situation

14 Why a Robustness Analysis The imperfection of the data set obtained should be properly considered in decision analysis Robustness analysis should consider all plausible data sets in order to identify a robust ranking of plausible good decisions (COAs) A specific set of data instantiates a potential realization of the model of decision.

15 Robustness [Kouvelis and Yu, 1997] From the point of view of the optimality The solution of a mathematical program is qualified as robust, if it remains in neighbourhood of the optimum for all plausible data sets of the model Generalisation from optimality to best compromise. Since the approach of robustness is crucially based on the process of generation of plausible data sets, it requires a good knowledge of the environment in which the decision take place

16 Robust ranking approach proposed Three critical steps were identified for robustness analysis in the context of COAs comparison: an approach to model all the data sets that instantiate the decision-maker s preferences, which are not so well known ; a method to aggregate the pre-orders generated from each data set; a robustness criterion suited for the decisionmaking situation.

17 Identification of Possible Data Sets Plausible Coefficients of relative importance of the Criteria Plausible Preference Modelling Thresholds Possible Data Sets Filtering Plausible Data Sets Plausible Data Sets

18 Coefficients of relative importance of the criteria (CRIC) Identification of intervals [π 1 (),π2 () ] 0 < π < 1, = 1 n based on decision-maker s intervals,..., based on decision-maker s explicit values

19 CRIC Based on decision-maker s explicit values [ π π ], Κ,[ π, π ], Κ,[ π π ] ( 1), (1) ( ) ( ) ( n), ( n) with π ( n) 0, π (1) 1, π ( ) π ( + 1) and π ( ) π ( 1) = 2,..., n 1 g,..., g,..., g ( 1) ( ) ( n) g g ( n) () 1 is the least important criterion is the most important criterion

20 CRIC Based on decision-maker s explicit values (2) Normalized with: π ( ) = 1 π + π ( ) ( ) 2 2

21 Ex oequo Intervals Process ex oequo as one (block) g () 1 ( g g g ( 2) ( 2) ( 3) g ( 5), g, g ), g ( 4) ( 4) ( 4) 1 2 [ π, π ] () 1 () [ π ( 2), π ( 2) ], π ( 2), π ( 2) 1 2 [ π, π ] ( 3) ( 3) [ ] [ π ( 4), π ( 4) ], π ( 4), π ( 4) 1 2 [ π, π ] ( 5) ( 5) 1 2 [ ], [ π ( 4), π ( 4) ] g,..., g,..., ( 1) ( ) ( n) [ π π ],,[ π, π ], Κ,[ π π ] 1 2 Κ with π ( n ) 0 and π (1) 1 ( 1), (1) ( ) ( ) ( n), ( n) g g g ( n) () 1 is the least important criterion is the most important criterion

22 CRIC Based on a Pre-Order of Importance Consider to have 6 data sets π 1 the c.r.i. are equally balanced π 6 the c.r.i. are decreasing from 1 to 1/n π (1) π (2). π (n) with π > 0 and, π n = 1 = 1 Reduce the values by slices on a basis of (n-2)/4 Example with 15 criteria :

23 Indifference Thresholds [ q, q ], with q 0 DM interval Otherwise DM value ' q Default value q = 0.15X ' E 80% and 60%

24 Preference Thresholds 2 [ p p ] p , q and p E DM interval Otherwise DM value ' p Default value 80% and 60% p ' = 0.25E ( v q )

25 Veto Thresholds [ v v ] with v p v > DM interval Otherwise DM value, v Default value v = 0.25E π 80% and 60%

26 Filtering Plausible Data Sets To reduce the number of data sets For each interval, use only 3 data First value, middle one, last one Treatment of parameters as groups or blocks

27 Discussion Characteristics of the proposed military decisionmaking model A=(a 1,,a i,,a m ); Λ/C=(g 1,,g,,g n ); E=(e i =g (a i ), i=1,...,m; =1,...,n); M=(π, v (e i ),q (e i ),p (e i ), i=1,...,m; =1,...,n); and a multicriterion method, PAMSSEM, within the framework of the ranking problematic. given m alternatives and n attributes/criteria

28 Discussion Identification of possible values for: coefficient of relative importance (π ) discrimination thresholds indifference (q ) preference (p ) veto thresholds (v ) Identification of plausible data sets

29 Conclusion Robust results should be less influenced by the imperfection of the data occurring in the evaluations of the courses of actions as well as in the instantiation of parameters representing decision-maker s preferences during the modeling process of a military situation Considering all plausible information that might represent the decision-making context Not constrained to a single data set Robustness concept should be generalised to other information/knowledge analysis methodologies e.g.; IPB and Enemy s estimates

30