Propagation of Uncertainty in a Maritime Risk Assessment Model utilizing Bayesian Simulation and Expert Judgment Aggregation Techniques

Transcription

1 Presentation September 24 Rutgers University 1 Propagation of Uncertainty in a Maritime Risk Assessment Model utilizing Bayesian Simulation and Expert Judgment Aggregation Techniques V. Dinesh (VCU),T.A. Mazzuchi (GWU), J.R.W. Merrick (VCU), A. Singh (GWU), P. Szwed (GWU) and J. Rene van Dorp (GWU) This material is based in part upon work supported by the National Science Foundation under Grant Nos. SES and SES Any findings, opinions, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

2 Presentation September 24 Rutgers University 2 1. ORGANIZATION Briefly review Bayesian Simulation of San Francisco Bay Exposure Analysis. Discuss in detail Bayesian methodology for assessing relative accident probabilities and their uncertainty using paired comparisons to elicit expert judgments. Approach is illustrated using expert judgment data elicited for The Washington State Ferry Risk Assessment in Propagate simulation uncertainty and expert judgment uncertainty in a proof of concept case study using WSF expert judgment and SF bay exposure simulation

3 Presentation September 24 Rutgers University 3 2. WHY ADDRESS UNCERTAINTY? ISM Fleet Wide MRRT>6 MRRT 1-6 Implement all Mechanical Failure Reduction Measures Fleet Wide MRRT -1 Implement High Speed Ferry Rules and Procedures Impliment Visibility Restrictions Implement Traffic Separation for High Speed Ferries Implement Traffic Control for Deep Draft Traffic Increase time available for Response -2% -15% -1% -5% % 5% 1% Example of WSF Risk Assessment Results

4 Presentation September 24 Rutgers University 4 2.E+8 Alternative 1 1.8E+8 Expected # Interactions Per Year 1.6E+8 1.4E+8 1.2E+8 1.E+8 8.E+7 6.E+7 4.E+7 2.E+7 Base Case Alternative 3 Alternative 2.E+.E+ 2.E+5 4.E+5 6.E+5 8.E+5 1.E+6 1.2E+6 1.4E+6 # Transits Per Year Example of SF Bay Exposure Assessment Results

5 Presentation September 24 Rutgers University 5 One problem with the representations in figures above is the apparent finality of the results. The decision-maker is led to believe that the results are definitive and are in no way uncertain. The National Research Council performed a peer review of the Prince William Sound Risk Assessment (the father and grand father risk assessment of the Washington State Ferry Risk Assessment and San Francisco Bay Exposure Assessment, respectively) and concluded that the underlying methodology shows "promise" to serve as a systematic approach for making risk management decisions for marine systems, but uncertainty in results needs to be addressed. "Risk management should answer whether evidence is sufficient to prove specific risks and benefits" (A. Elmer, President, SeaRiver Maritime, Inc. in National Research Council, 2). "Since the truth is, we always have uncertainty, we say that speaking in probability curves is telling the truth". (see, e.g., Kaplan, 1997, p. 412)

6 Presentation September 24 Rutgers University 6 1. BAYESIAN SIMULATION IN A NUTSHELL Bayesian simulation differs from classical simulation analysis in that probability distributions are used to represent the uncertainty about model parameters rather than point estimates and confidence intervals. Such treatment is applied to both random inputs to the model and the outputs from the model. Bayesian San Francisco Bay Exposure Analysis: 1. Bayesian Poisson process models of traffic arrivals were created for all 5,277 arrivals processes (excluding the Ferries since these run on a tight schedule). 2. For each arrival process a Gamma Prior Distribution was postulated for the arrival rate - of the exponential distributed inter arrival time. 3. For each arrival process a Gamma Posterior distribution for the arrival rate - was determined by a conjugate Bayesian analysis utilizing the inter arrival data for that process.

7 Presentation September 24 Rutgers University 7 4. In the simulation, for each arrival one first samples an arrival rate - from its Gamma distribution and next sample the inter arrival time from the exponential distribution utilizing the sampled arrival rate -, etc. HENCE, BAYESIAN SIMULATION TECHNIQUES ADDRESS BOTH: THE RANDOMNESS WITHIN THE SYSTEM UNDER CONSIDERATION (I.E. ALEATORY UNCERTAINTY) AND LACK OF KNOWLEDGE ABOUT THE SYSTEM (I.E. EPISTEMIC UNCERTAINTY)

8 Presentation September 24 Rutgers University 8 5. As our output data is in the form of a count (i.e. number of interactions per grid cell or total number of interaction), this number of vessel interactions can be naturally modeled using a Poisson distribution with rate -, with a conjugate prior gamma distribution with shape α and scale #. 6. For = replications the posterior distribution of the number of vessel interactions is again a Poisson distribution with rate -, with a conjugate posterior gamma = distribution with shape α 8 and scale # =, where is the number of 4œ" counts observed in the j-th replication and = is the total number of (independent) replications of the simulation. 7. Our predictive distribution for the number of interactions (per grid cell) is then a Poisson Gamma distribution. T<Ð\ œbñ œ = > Ðα 8 BÑ # = α œ" " 4œ" Š = # = " BÐ# = "Ñ > ÐBÑ> Ðα 8 Ñ See, e.g., Bernardo and Smith (1994), Bayesian Theory, Wiley = 4œ" 4 B, B œ!ß "ß #ß ÞÞÞ

9 Presentation September 24 Rutgers University 9 3. SF BAY EXPOSURE ASSESSMENT RESULTS WITH UNCERTAINTY After 1 day of simulation Posterior 5-th percentile Posterior 5-th percentile Posterior 95-th percentile

10 Presentation September 24 Rutgers University 1 3. SF BAY EXPOSURE ASSESSMENT RESULTS WITH UNCERTAINTY After 5 years of Simulation Posterior 5-th percentile Posterior 5-th percentile Posterior 95-th percentile

11 Presentation September 24 Rutgers University AGGREGATE SF BAY EXPOSURE ASSESSMENT RESULTS WITH UNCERTAINTY A 6.42E+5 B 9.76E+5 # Yearly Situations 6.41E+5 6.4E+5 # Yearly Situations 9.75E E E+5 Base Case 9.73E+5 Alternative 3 C 7.2E+6 D 1.26E+7 # Yearly Situations 7.19E E+6 # Yearly Situations 1.26E E E E+6 Alternative E+7 Alternative 1

12 Presentation September 24 Rutgers University AGGREGATE SF BAY EXPOSURE ASSESSMENT COMPARISON RESULTS WITH UNCERTAINTY 1.4E+7 1.2E+7 1.E+7 # Yearly Situations 8.E+6 6.E+6 4.E+6 2.E+6.E+ Base Case Alternative 3 Alternative 2 Alternative 1 BOX PLOTS A-D OF PREVIOUS SLIDE IN A SINGLE GRAPH

13 Presentation September 24 Rutgers University 13

14 Presentation September 24 Rutgers University CONCLUSION OF BAYESIAN SIMULATION OF SF BAY MTS IN A NUTSHELL The uncertainty in the random arrival patterns of non-ferry traffic does not seem to affect in this particular study the conclusions regarding the different expansion scenarios (Base Case, Alternative 3, Alternative 2 and Alternative 1) from an Exposure Perspective. (which could be explained by the fact that interactions are predominantly FERRY to FERRY interactions which run on a tight schedule). For more detailed information see: Merrick, J. R. W., J. R. van Dorp and V. Dinesh (23). Assessing Uncertainty in Simulation Based Maritime Risk Assessments. Submitted to Risk Analysis. Merrick, J. R. W., V. Dinesh, A. Singh, J. R. van Dorp and T. Mazzuchi (23). Propagation of Uncertainty in a Simulation-Based Maritime Risk Assessment Model Utilizing Bayesian Simulation Techniques. Winter Simulation Conference 23 Proceedings.

15 Presentation September 24 Rutgers University ACCIDENT PROBABILITY ASSESSMENT USING BAYESIAN PAIRED COMPARISON ELICITATION An important class of elicitation techniques consists of the psychological scaling models that use the concept of paired comparisons. Origins can be traced back to Thurstone's (1927) and Bradley (1953)). Another popular paired comparison elicitation technique is called the Analytical Hierarchy Process (AHP) developed by Saaty (1977, 198). The AHP Process is primarily used for the construction of value functions ZÐ\ Ñinvolving multiple contributing factors \ œð\ß\ßáß\ñ(see, e.g. Foreman and Selly (22)). " # : The popularity of the paired comparison method can perhaps be contributed to the observation that experts are more comfortable making comparisons rather than directly assessing a quantity of interest.

16 Presentation September 24 Rutgers University 16 To the best of our knowledge, Pulkkinen (1993, 1994) was first to introduce a Bayesian paired comparison aggregation method for the elements of a multivariate random vector " œð" " ß" # ßáß" : Ñby multiple experts. Pulkkinen's (1993, 1994) exposition is mainly theoretical and limited to a discussion of mathematical propertiesþ Similar to the AHP process, we are interested in the functional relationship between contributing factors \ œð\ß\ßáß\ñ " # : and an accident probability T <ÐE--3./8>l M8-3./8>ß \) defined by T <ÐE--3./8>lM8-3./8>ß \ ) œ T IB: ˆ X " \. Ð"Ñ \ œð\ß\ßáß\ñ " # : describes a system state during which an incident (e.g. a mechanical failure) occurred. The accident probability model Ð"Ñ resembles the well-known proportional hazards model originally proposed by Cox (1972) and builds on the assumption that accident risk behaves exponentially rather than linearly with changes in covariate values.!

17 Presentation September 24 Rutgers University 17 Our goal is to establish the uncertainty distribution of the accident probability T <ÐE--3./8>lM8-3./8>ß \ ) in entirety rather than a point estimate. Organizational Factors Incident Accident Consequences Situational Factors Sequence Influence

18 Presentation September 24 Rutgers University 18 Table 1. Description of " contributing factors to T<ÐE--3./8> M8-3./8>, \) in WSF Risk Assessment H/=318+>398 H/=-<3:>398 H3=-</>3D+>398 \" FR_FC Ferry route-class combination #' \# TT_1 1st interacting vessel type "$ \ $ TS_1 Scenario of 1st interaction % \% TP_1 Proximity of 1st interaction F38+<C \& TT_2 2nd interacting vessel type & \' TS_2 Scenario of 2nd interaction % \( TP_ 2 Proximity of 2nd interaction F38+<C \) VIS Visibility F38+<C \* WD Wind direction F38+<C \"! WS Wind speed G98>38?9?= : : \ Ò!ß "Ó, " and T! Ð!ß "ÑÞ The covariate \ 3ß 3 œ "ß á ß : are normalized so that \ 3 œ " describes the "worst" case scenario and \ 3 œ! describes the "best" case scenario.

19 Presentation September 24 Rutgers University 19 Naval Vessel Roll-On/Roll-Off Other Tanker Refrigerated Cargo Bulk Carrier Container Freight Ship Tug Boat/Barge Passenger High Speed Ferry Small Ferry Large Ferry Vessel Class.% 2.% 4.% 6.% 8.% 1.% Relative Scale of Concern Constructed Covariate Scale for Interacting Vessels

20 Presentation September 24 Rutgers University 2 Question: Situation 1 Attribute Situation 2 Super Ferry Class - SEA-BAI Ferry Route - Naval Vessel 1st Interacting Vessel - Crossing the bow Traffic Scenario 1st Vessel - 1 to 5 miles Traffic Proximity 1st Vessel - Deep Draft 2nd Interacting Vessel - Crossing the bow Traffic Scenario 2nd Vessel - 1 to 5 miles Traffic Proximity 2nd Vessel - more than.5 mile Visibility less than.5 mile Along Ferry Wind Direction - 4 knots Wind Speed Situation 1 is worse <====================X====================> Situation 2 is worse An example question appearing in one of the questionnaires used in the WSF risk assessment " # X " # T Ð\,\ " Ñ œ IB: " ˆ \ \ Ò!ß ÓÞ Ð#Ñ P91 " # T Ð Ñ X œ ˆ " # \,\ " " \ \ Ð ß Ñ Ð$Ñ

21 Presentation September 24 Rutgers University THE LIKELIHOOD OF A SINGLE EXPERT'S RESPONSE T <ÐE--3./8>lM8-3./8>ß \ ) ] 4 œ Experts response to ratio ß T <ÐE--3./8>lM8-3./8>ß \ ) ^4 œ P91] 4, 4 œ "ßáß8. The response of the expert to such a question is uncertain and will assumed to be normal distributed such that Ð^. ß <Ñ µ R Ð. ß <Ñß < œ "Î5 Ð%Ñ # 4 " 4 # X " #. 4 4 ; 4 \ 4 \ 4 œ; ", œ( ) Ð&Ñ < ^ ÐD4Ñ º È< /B: ÐD4 4 Ñ # Ð'Ñ 4 œ.. #

22 Presentation September 24 Rutgers University 22 " # Expert answers 8 paired comparison questions defined by ; 4 œð\ 4 \ 4 ), 4œ"ßáß8, Define U to be the : 8 matrix and m to be the vector with log responses of expert UœÒ; 1, á, ; 8Ó, m œ ÐD" ßáßD8Ñ. Ð(Ñ Likelihood of an expert responding m to questionnaire Uß may be derived from Ð'Ñ as being proportional to 8 _m Ð l ", <, UÑ º < /B: < # œ Ð- 2, X T " " E" Ñ Þ Ð*Ñ # where E œ X ; ; ;, œ ; D ; - œ D Ð"!Ñ œ" 4œ" 4œ" If columns of U span : the matrix E can be shown to be symmetric ß positive definite and henceforth invertible. 4 #

23 Presentation September 24 Rutgers University PRIOR DISTRIBUTION To allow for a conjugate Bayesian analysis a multivariate normal/gamma prior is proposed for the joint distribution of Ð" ß<Ñsimilar to the one described in West and Harrison (1989). / α # α $ # " < α / Ð < lα/, Ñ œ < # α /B:Ð /Ñ, i.e. K+77+Ð ß ÑÞ Ð""Ñ > Ð Ñ # # # # : < $ Ð " l < Ñ º < # /B: œ Ð" 7Ñ X? Ð" 7Ñ, i.e. QZ RÐ7ß <? ÑÞÐ"#Ñ # Hence, the joint prior distribution on Ð" ß <Ñ follows from Ð""Ñ and Ð"#Ñ to be α : < $ Ð " ß < Ñ º < # " < /B:Ð < # X /B: œ Ð" Ñ Ð" Ñ. Ð"$Ñ # / Ñ # 7? 7

24 Presentation September 24 Rutgers University 24 The marginal distribution of " may be derived from Ð"%Ñß yielding α : # " $ X Ð " Ñ º " Ð" 7Ñ? Ð" 7Ñ Ð"%Ñ / and is recognized as a :-dimensional multivariate > -distribution with α degrees of freedom, location vector 7 and precision matrix. /? α From Ð"%Ñ and Ð$Ñ follows that the log-relative probability P91 T Ð\ ",\ # " Ñ has a prior > -distribution with mean and precision X X 7 ˆ " # " # " # \ \, α ˆ \ \? ˆ \ \ Ð"&Ñ / 9.1. Prior Parameter Specification A prior chi-squared distribution with α degrees of freedom (equivalent to a α / gamma distribution K+77+Ð ß Ñ with / œ "Ñ and IÒ<lα/ ß =1 Ó œ α. # #

25 Presentation September 24 Rutgers University 25 The prior parameter α will be set equal to the reciprocal of the variance of an expert responding at random and depends on the scale that is used in the paired comparison questions to collect the expert responses. * # " α œ IÒ<lα ß / =1Ó œ š # ÖP91Ð5Ñ!Þ$)!$%"Þ Ð"'Ñ "( 5œ# For distribution of Ð" l<ñ we may select a location vector and the unit precision matrix " g 7 œ Ð!ß á ß!Ñ X,? œ Î ä Ñ, Ð"(Ñ Ïg " Ò as long as the prior distribution on the relative accident probabilities Ð#Ñ are flatþ The pdf of the relative accident probability in our previous question is a log-> distribution (see, e.g., McDonald and Butler (1987)) with prior parameters X 7 ˆ " # " # " # \ \ œ!, α œ!þ$)!$%"ß / œ " ß $ œ ˆ \ \? ˆ \ \ œ %. 33 X

26 Presentation September 24 Rutgers University 26 PDF PDF A B C P( X, X β ) r Parameter Index Prior on Ð" ß<Ñ and TÐ\, \ " Ñ of previous question 5% " #

27 Presentation September 24 Rutgers University 27 " # " The prior median of TÐ\, \ Ñ equals "(indicating indifference in " # collision likelihood between system states \ and \ ). " # " A &!% credibility interval of TÐ\, \ Ñin the figure above equals Ò!Þ")"ß &Þ&"&Ó. A (&% credibility interval of TÐ\ ", \ # " Ñequals & % Ò#Þ!"# "! ß %Þ*(" "! Ó (which is quite wide)þ Table 2. Interaction Variables associated with the contributing factors in Table 1. R +7/ H/=-<3:>398 H3=-</>3D+>398 \" 1 FR_FC TT_1 Interaction "$ \"# FR_FC TS_1 Interaction "$ \"$ FR_FC VIS Interaction % \"% TT_1 TS_1 Interaction F38+<C \"& TT_1 VIS Interaction "$ \ TS_1 VIS Interaction % "'

28 Presentation September 24 Rutgers University POSTERIOR ANALYSIS Applying Bayes theorem utilizing the likelihood Ð*Ñ ß the prior distribution Ð"$Ñ and it follows that the posterior distribution # Ð" ß<lmß UÑis proportional to α 8 $ Ð " ß < l m ß U Ñ º < # " < X /B: œ Š " - 7? 7 Ð")Ñ # : < X </B: # œ Š #? X, 7 " " E? ". #??? Defining? to be? œe? and implicitly defining 7 satisfying X?? X,? 7 " œ? 7 " Ð"*Ñ for all " ß it follows that,???? 7 œ? 7 Í 7 œ Š? " Š,? 7. Ð#!Ñ

29 Presentation September 24 Rutgers University 29? Utilizing Ð#!Ñ and? œ E? we derive from Ð")Ñ that # Ð" ß<lm ß UÑ º < α 8 # " < /B: œ Š " - X????? # : < # X??? < # /B: œ " 7? " 7 Þ Ð#"Ñ X From Ð#"Ñ it follows that Ð" lm ßUѵQZRÐ ß<? Ñwhere 7?? Ú 8? Ý? œ ; 4; 4 X? 4œ" Û?? Ý Š? " 8 7 œ Š ; 4D4? 7 Ü 4œ"?? α / m # # and Ð<l ß U Ñ µ K+77+Ð ß Ñ with Ú? α œ α 8 8 X Û / œ / D4 7? 7 7? 7 Ü? # X??? 4œ" Ð$!Ñ Ð$"Ñ

30 Presentation September 24 Rutgers University EXAMPLE FROM WSF RISK ASSESSMENT 8 Experts were selected amongst WSF captains and WSF first mates who had extensive experience with all 13 different ferry routes over an extended period of time (more than 5 years). Combination of the responses of these ) experts follows naturally by exploiting the conjugacy of the analysis in Section 3, 4 and 5 through sequential updating. Table 3. Expert Response to Previous Paired Comparison Question Expert Index Response During the WSF risk assessment in 1998 expert responses were aggregated by taking geometric means of their responses and using them in a classical log linear regression analysis approach to assess relative accident probabilities given by Ð#Ñ. Classical point estimates for the parameters " 4 ß 4 œ "ß á ß "' will be compared to their Bayesian counterparts following our Bayesian aggregation method.

31 Presentation September 24 Rutgers University 31 Expert were instructed to assume that a navigation equipment failure had occurred on the Washington State Ferry and were next asked to assess how much more likely a collision is to occur in Situation 1 (good visibility in previous question) as compared to Situation 2 (bad visibility in previous question) taking into account the value of all the contributing factors. Total of 6 Questions. The questions were randomized in order and were distributed evenly over the "! contributing factors in Table 1 (i.e. ' questions per changing contributing factor) The elements Eß, and - of the likelihood given by Ð" Ñ E œ E"" E"# E E #" ## Ð$#Ñ where E "" is a "! "! diagonal matrix with diagonal elements Ð%Þ&'ß %Þ$$ß #Þ)*ß 'ß "Þ&ß #Þ%%ß 'ß 'ß 'ß!Þ$(&Ñ Ð$$Ñ and associated with the contributing factors \ßáß\ " "!. (The matrix E "" in Ð$#Ñ is a diagonal matrix since the paired comparison scenarios \ " and \ 2 only

32 Presentation September 24 Rutgers University 32 differed in one covariate (see,e.g., the previous question). The matrix E ## in Ð$#Ñ is a symmetric ' ' matrix with elements Ô $Þ%&!Þ$$! "Þ%%!Þ('!!Þ$$ $Þ%&!Þ%%!Þ$$! "!!Þ%% %Þ""! " #Þ$* "Þ%%!Þ$$! "Þ)*!Þ$'!Þ!) Ö Ù!Þ('! "!Þ$' $Þ!# # Õ! " #Þ$*!Þ!) # 'Þ'( Ø Ð$%Ñ and associated with the interaction effects \"" ßáß\ "'. Finally, the matrix X E œ E is a sparse "! ' matrix #" "# Ô " #Þ)#!!!!!!!! #Þ#'! #Þ"#!!!!!!! "Þ"$!!!!!! $Þ!'!!! #Þ"$!Þ&#!!!!!!! Ö Ù! "Þ!#!!!!! #!! Õ!! "Þ&'!!!! &Þ$$!! Ø with only positive elements associated with the contributing factors \ß\ß\ and \ that are included in the interaction effects \"" ßáß\ "' Þ " # $ ) Ð$&Ñ

33 Presentation September 24 Rutgers University FR-FC TT TS TP TT TS TP VIS WD WS FR-FC * TT FR-FC * TS FR-FC * VIS TT1*TS TT1*VIS TS1*VIS Summary of Individual Expert Response for 8 WSF experts in terms of 3-th element of the vector, Ðcf. Ð""Ñ for each of the contributing factors \ 3ß 3 œ "ß á ß "! in Table 1 and interaction effects \ 3ß 3 œ ""ß á ß "' in Table 2.

34 Presentation September 24 Rutgers University 34 Table 4. Values for - (cf. Ð""ÑÑ for the ) individual experts. Expert Index "%*Þ!( *&Þ#) &&Þ(% "%(Þ*$ ")&Þ(" "((Þ$! "%(Þ"# %%Þ*% Posterior Analysis?? α / # # The resulting posterior parameters for the precision < µ K+77+Ð ß Ñ are?? α œ %)!Þ$)ß / œ &$!Þ*& Ð$'Ñ The posterior distribution of the parameter vector " is a multivariate > distribution with location vector 7? and precision matrix α?? where?,? /?? ß α / are given by Ð$'Ñ,?? œ? )E and location vector 7? is depicted in the following figure.

35 Presentation September 24 Rutgers University 35 Point Estimates of Covariate Parameters TS_1*VIS TT_1*VIS TT_1*TS_1 FR-FC*VIS FR-FC*TS_1 FR-FC*TT_1 WS WD VIS TP_2 TS_2 TT_2 TP_1 TS_1 TT_1 FR-FC Bayesian Point Estimate Classical Point Estimate Comparison of Bayesian and Classical Point Estimates of the parameters " 3 ß 3œ"ßáß"'Þ

36 Presentation September 24 Rutgers University 36 It can thus be concluded that traffic proximity of the first and second interacting vessel (\% and \(, respectively), traffic scenario of the second interacting vessel \( and wind speed \"! are the largest contributing factors to accident risk. In addition, the manner in which the first interacting vessel approaches the ferry route - ferry class combination (\"# Ñ, i.e. crossing, passing or overtaking, and in what visibility conditions Ð\ "' Ñ are the largest interacting factors. A remarkable agreement should be noted between the Bayesian and classical point estimates provided in the figure above, except for a discrepancy associated with the contributing factor WS (Wind Speed). The next figure displays the posterior distribution of the relative probability TÐ\ ", \ # " Ñassociated with our previous pair wise comparison question.

37 Presentation September 24 Rutgers University 37 PDF PDF A B Parameter Index r C P( X, X β ) 99% " # Posterior on Ð" ß<Ñ and TÐ\, \ " Ñ of Previous Question.

38 Presentation September 24 Rutgers University 38 " # " The median point estimate of T Ð\, \ Ñ equals %Þ*%Þ Hence, Situation 2 in Figure 3 is approximately 5 times more likely to result in a collision than Situation 1 given that a navigation equipment failure occurred on the ferry. " # " Compare the &!% posterior credibility interval of T Ð\, \ Ñ of Ò%Þ()ß &Þ"$Ó to the &!% prior one of Ò!Þ")ß &Þ&#Ó. In addition, the **% posterior credibility interval of Ò%Þ$$ß &Þ''Ó is indicated in the figure above which is remarkably narrow compared to the prior (&% credibility interval of & % Ò#Þ!"# "! ß %Þ*(" "! Ó Utilizing posterior distributional results for the parameter vector " credibility statements can be made for any arbitrary paired comparison. For example, setting Situation 1 in Ð#Ñ to the best possible scenario (\ " œ!) and Situation 2 to the worst possible scenario (\ # œ ") a **% credibility interval of T Ð\ ", \ # " Ñ equals Ò$""%#ß $'(%*ÓÞ Therefore, collision risk in the worst possible scenario differs at least by 4 orders of magnitude to that of the best possible scenario while taking uncertainty of the expert judgments into account.

39 Presentation September 24 Rutgers University COMMENTS ON EXPERT JUDGMENT METHOD Bayesian aggregation method has been developed using responses from multiple experts to a paired comparison questionnaire to assess the distribution of relative accident probabilities. The classical analysis conducted during the WSF risk assessment only resulted in point estimates of relative accident probabilities. For more detailed information see: P. Szwed, J. R. van Dorp, J. R. W. Merrick, T. A. Mazzuchi and A. Singh (24). A Bayesian Paired Comparison Approach for Relative Accident Probability Assessment with Covariate Information. European Journal of Operational Research, Vol. 161 (1), pp PREVIOUS EXPERT AGGREGATION METHOD UTILIZED SEQUENTIAL UPDATING WHICH ESSENTIALLY MEANS THAT GIVEN THE PARAMETER VECTOR " AND THE PRECISION < THE EXPERTS ARE STATISTICALLY INDEPENDENT.

40 Presentation September 24 Rutgers University 4 THE LATTER ASSUMPTION CAN BE RELAXED! ESSENTIALLY BY UTILIZING A MULTIVARIATE BAYESIAN REGRESSION APPROACH TO THE PAIRED COMPARISON QUESTIONS INVOLVING A PRIOR MULTIVARIATE NORMAL ON (" D) WHERE DIS THE INVERSE VARIANCE COVARIANCE MATRIX AND A WISHART PRIOR OR D. (see, e.g., Press, S. J Applied Multivariate Analysis Using Bayesian and Frequentist Methods and Inference. 2nd Edition. Robert E. Krieger Publishing Company, Malabar, Florida). For more detailed information on our specific application see: Merrick, J. R. W., J. R. van Dorp and A. Singh (23). Analysis of Correlated Expert Judgments from Pairwise Comparisons. Re-submitted to Decision Analysis, September 24, First Revision. A surprising result is that the incorporation of dependence here results in a reduction of the predictive variance of the elements of the parameters vector "

41 Presentation September 24 Rutgers University Density of Beta INDEPENDENCE AMONGST EXPERTS Density of Beta DEPENDENCE AMONGST EXPERTS -3 FR-FC TT_1 TS_1 TP_1 TT_2 TS_2 TP_2 VIS WD WS FR-FC *TT_1 FR-FC *TS_1 FR-FC *VIS TT_1 *TS_1 TT_ *VIS TS_1 *VIS 9% Creditibility Intervals of the elements of the parameter vector " of T <ÐE--3./8>lM8-3./8>ß \ œ T IB: ˆ X " \ )!

42 Presentation September 24 Rutgers University PUTTING IT ALL TOGETHER IN A PROOF OF CONCEPT CASE STUDY # Yearly Accidents Base Case Alternative 3 Alternative 2 Alternative 1

43 Presentation September 24 Rutgers University 43 Whereas an almost certain ranking in terms of the expected yearly number of situations, this is not true for t he expected yearly number of accidents. A.8 B.8 # Yearly Accidents.7.6 # Yearly Accidents Base Case.5 Alternative 3 SEPARATE BOX PLOTS A-B OF PREVIOUS SLIDE The box plots for the Base Case and Alternative 3 show that the range of their distributions do indeed overlap and the best we can say is that Alternative 3 stochastically dominates the Base Case in the sense that their cumulative distribution functions do not cross.

44 Presentation September 24 Rutgers University 44 Reason for Stochastic Dominance and not Deterministic Dominance of the Alternative 3 and the Base Case is a reduction in the average accident probability per occurring situation. Average Probability of an Accident across Occurring Situations 1.4E-1 1.2E-1 1E-1 8E-11 6E-11 4E-11 2E-11 Base Case Alternative 3 Alternative 2 Alternative 1

45 Presentation September 24 Rutgers University 45 Reason for a reduction in the average accident probability per occurring situation is a reduction in Alternative 3 of the number of second interacting vessels within a 1 mile distance. (Recall that Traffic Proximity of second interacting vessel was a dominant factor in the accident probability in a given interaction.) 1% 9% < 1 mile > 1 mile No Other Vessel Percentage of Situations 8% 7% 6% 5% 4% 3% 2% 1% % Base Case Alternative 3 Alternative 2 Alternative 1

46 Presentation September 24 Rutgers University 46 TENTATIVE OBSERVATION: The reduction in accident probability per interaction could be an indication of what might be achieved when looking at the SF Bay Ferry operation as a System of interconnected Ferry Routes and by designing a Comprehensive Ferry Schedule that aims to reduce the number of interactions and the number of vessels that are interacting in given situation. For more detailed information on this proof of concept case study see: Merrick, J. R. W. and J. R. van Dorp (24). Speaking the Truth in Maritime Risk Assessment. Submitted to Management Science, August 24. QUESTIONS?

47 Presentation September 24 Rutgers University MARITIME RISK ASSESSMENT LINKS Faculty Home Page of J. Rene van Dorp: and Faculty Home Page of Jason R.W. Merrick: Available for downloading: Presentations, Journal Papers, Proceedings, Reports SF Bay Simulation Movies