Aggrega?on of Epistemic Uncertainty

Transcription

1 Aggrega?on of Epistemic Uncertainty - Certainty Factors and Possibility Theory - Koichi Yamada Nagaoka Univ. of Tech. 1 What is Epistemic Uncertainty? Epistemic Uncertainty Aleatoric Uncertainty (Sta?s?cal / Objec?ve Uncertainty) related to frequency - cogni?ve uncertainty caused by incomplete knowledge / lack of informa?on - subjec?ve uncertainty Examples of Epistemic Uncertainty - Q1: Gravity accelera?on in this room : about 9.8 m/s 2 - Q2: whether the suspect arrested is the real murderer or not. The uncertainty contained in the answers cannot be represented by frequency. It is uncertainty considered as a degree of our belief. 2 Compu?ng 1

2 Which Uncertainty Does Probability Represent? Originally, probability had been a measure to represent "frequency." In 1740s, Thomas Bayes considered a way to deal with degrees of belief in the framework of Probability theory, which is called Bayesian Probability or subjec?ve probability. Examples of Subjec?ve probabili?es : It will rain tomorrow at 50%. Our team will win tomorrow at 99%. These percentages are not frequencies, because tomorrow will come just once. They are our beliefs. Probability can be used both for frequency (Aleatoric uncertainty) and for degrees of belief (Epistemic uncertainty). 3 Other Theories for Represen?ng Uncertainty Possibility Theory - a theory related to an adjec?ve, "Possible" Dempster-Shafer theory of Evidence - a generalized theory of uncertainty Certainty Factor - Uncertainty representa?on employed in MYCIN (1974) Fuzzy set theory - vagueness contained in concepts and words Rough Set theory - indiscernibility and approxima?on due to our limited knowledge Mul?-valued logics - truth between "completely true" and "completely false" Note: These are all theories to deal with Epistemic Uncertainty, which suggests there are many aspects in Epistemic Uncertainty. è focus on the degree of beliefs 4 Compu?ng 2

3 What is Important for Dealing with Epistemic Uncertainty? Capability to deal with "ignorance" / "unknown situa?on" is important. Example: Suppose there is a gang group in a small town, and your old friend Tom has joined it. One day, a murder happened in the town. There was perfect evidence that one of the gang members did it. No other informa?on is given. How do you represent uncertainty that "Tom is the murderer"? Probability theory P(Tom) =1/ n n : the number of the gang members, but we do not know the exact number. Probability cannot represent the uncertainty of this situa?on. 5 Representa?on in Other Theories Possibility theory: an uncertain situa?on is represented by a pair of possibili?es; π (Tom) =1.0 π (Tom) =1.0 : Possibility that Tom is the murderer is 1.0. : Possibility that Tom is NOT the murderer is π 0.0 Dempster-Shafer theory of Evidence: uncertainty is represented by two measures; Pl(Tom) = 1.0 Bel(Tom) = Pl,Bel 0.0 Certainty Factor Model CF(Tom) = CF : Plausibility that Tom is the murderer is 1.0. : Belief that Tom is the murderer is : perfect affirma?on 1 : perfect nega?on 0 : unknown or no informa?on 6 Compu?ng 3

4 Aggrega?on of Epistemic Uncertainty In everyday-decision making, we frequently gather mul?ple pieces of uncertain informa?on, and aggregate the informa?on. - Which beach resort shall we go to during the next vaca?on? - Which job should I choose among the mul?ple offers? - Which is telling the truth, President Trump or the New York Times? We need to gather and aggregate much uncertain informa?on to answer these ques?ons. - Aggrega?on is one of the most important informa?on processing for Epistemic Uncertainty. - Many applica?ons need informa?on aggrega?on in decision-making, affec?ve informa?on processing, sensor fusion, flexible informa?on retrieval, etc. There are only a few theories that provide a standard aggrega?on func?on. - Dempster-Shafer Theory of Evidence : Dempster's rule of combina?on - Certainty Factor Model 7 Certainty Factor Model CF was devised and used to represent uncertainty of a hypothesis given some evidence in stead of Probability in a famous Expert System MYCIN. because, - Probability cannot express the unknown situa?on (ignorance). - No standard way to aggregate mul?ple probability distribu?ons derived from mul?ple pieces of evidence. The CF model was evaluated as "Prac?cal" by many prac??oners, but was also cri?cized harshly by theore?cians, blaming it is theore?cally wrong. - There was no sound interpreta?on of the CF model in the framework of Probability theory. 8 Compu?ng 4

5 Original Defini?on of Certainty Factors CF of hypothesis h given evidence e C f (h,e) [ 1, +1] +1 : perfect affirma?on 1 : perfect nega?on 0 : neither is supported (unknown, no evidence) MB(h,e) : degree that belief in h is revised by e toward affirma?on MD(h,e) : degree that belief in h is revised by e toward nega?on - We do not adopt the defini?on, because it was proved that the defini?on is not consistent with the aggrega?on func?on of CFs. 9 Aggrega?on Func?on of CF Model Let x (y) be CF of hypothesis h given evidence ex (ey). x = Cf (h, e x ) y = Cf (h,e y ) Then, the Aggrega?on (combina?on) func?on is given as follows; The aggregated CF could be interpreted as follows; f M (x, y) = Cf (h, e x e y ) The equa?on is commuta?ve and associa?ve. So, aggrega?on results are not dependent on the order of sequence, when there are mul?ple CFs. 10 Compu?ng 5

6 New Sound Interpreta?on of CFs with Possibility Theory Koichi Yamada: Aggrega?on of Epistemic Uncertainty: A New Interpreta?on of the Certainty Factor with Possibility Theory and Causa?on Events, SCIS&ISIS 2018 (submioed) The rest of this presenta?on discusses a new sound interpreta?on of Certainty Factors using Possibility theory. A Certainty Factor Transformable to each other the both represent the same uncertainty A Possibility Distribu?on Examine new aggrega?on func?ons in the frame work of Possibility theory. - One of the aggrega?on func?ons is exactly the same as the one used in MYCIN. è This gives the theore?cal basis to the MYCIN's aggrega?on func?on. 11 Possibility is another scale to measure uncertainty with a value in [0, 1] similar to "Probability. According to L. A. Zadeh, Possibility Theory - Essen?ally, humans u?lize possibility rather than probability in decision-making. - Vagueness contained in Natural Language is principally possibilis?c. Possibility Seems appropriate for represen?ng epistemic uncertainty Some impressive statements about Possibility and Probability - What is impossible is improbable. (Zadeh) - What is possible can be improbable. (Zadeh) - What is improbable is not impossible, necessarily. (Zadeh) - What is probable must be possible. (D. Dubois and H. Prade) 12 Compu?ng 6

7 Possibility Measure Axioms of possibility measures Π( ) = 0 Π( U) = 1 Π(A B) = max(π(a),π(b)) Possibility Measure Possibility measure is defined in the similar way to Probability measure. Algebraic sum of Probability is replaced by Max opera?on in Possibility. Π : 2 U [0,1] U: the universal set P : 2 [0,1 ] A and B do not need to be disjoint. Proper?es A B Π( A) Π( B) c N( A) = 1 Π( A ) Necessity measure A is necessary = Not A is not possible Probability Measure P( ) = 0 U Axioms of probability measures P( U) = 1 P ( A B) = P( A) + P( B) Proper?es A B P( A) P( B) P(A) =1 P(A C ) if P(A B) = P(A) + P(B) P(A B) A B = 13 Possibility Distribu?on Both Probability and Possibility have distribu?on func?ons. Algebraic sum of Probability is replaced by Max opera?on in Possibility. Possibility distribu?on π :U [0,1] Proper?es π u ) = Π({ u }) ( i i Π(A) = Max u i A Π(U ) = Maxπ (u i ) u i U Π({u i }) = Maxπ (u i ) u i A = Max{π (u 1 ),π (u 2 ),...,π (u n )}=1 Probability distribu?on p :U [0,1] Proper?es p u ) = P({ u }) ( i i u A P ( A) = P({ u }) = p( u ) P( U) = p( ui ) i ui U i u A = p( u1 ) + p( u2) p( un ) = 1 i i A possibility distribu?on must be "normal." 14 Compu?ng 7

8 Condi?onal Possibility: Condi?onal Possibility Condi?ons and Independence are defined in a similar way despite that details are different. Algebraic product of Probability is replaced by Min opera?on in Possibility. Π ( B A) Condi?onal Probability: P( B A) Π(A B) = min(π(a),π(b A)) P(A B) = P(A) P(B A) Π( B A) = 1, Π if Π( A) = Π( A B) 0 ( A B), (1) When A is independent of B, Π(A B) = Π(A) (2) When B is independent of A, Π(B A) = Π(B) if Π( A) > Π( A B) (3) When (1) or (2) holds, A and B is non-interac?ve. Π(A B) = min(π(a),π(b)) P( A B) P( B A) =, if P( A) 0 P( A) A is independent of B B is independent of A. P(A B) = P(A) P(B A) = P(B) P(A B) = P(A) P(B) 15 Hypothesis and Opposite Hypothesis We introduce the Opposite Hypothesis k to a hypothesis h, which sa?sfies the following logical formulae. Which is the murderer, male or female? The hypothesis h and O-hypothesis k sa?sfy the following; is not tautology. is not contradic?on. h k male female unknown Note: Assuming the closed world assump?on, the asser?on of male (female) should be rejected, if there is no evidence for male (female). If we have no evidence both for male and female, we have to reject both asser?ons of male and female. represents "unknown" because of no evidence 16 Compu?ng 8

9 Causa?on Events Y. Peng and J. A. Reggia (1987) h : e - represents an event that evidence e supports hypothesis h k : d - represents an event that evidence d supports O-hypothesis k 17 Hypothesis, O-hypothesis, Causa?on Event Eh : the set of all pieces of evidence that possibly supports h. Ek : the set of all pieces of evidence that possibly supports k. Hypothesis h is true at least, one of possible pieces of evidence supports h, and no possible pieces of evidence supports k. If we define h and k above, they sa?sfy the formulae 18 Compu?ng 9

10 Condi?onal Causa?on Possibility CondiBonal CausaBon Possibility is possibility that evidence ei supports hypothesis h, only given the evidence ei.! represents that only ei is present and the others are not. Note : we assume is not contradic?on, even if. represent that evidence ei is not present (found) nor ej. This is possible, because we are considering the epistemic world. 19 Proposi?on Possibility that h is true only given evidence e is the same as the possibility that the evidence supports the hypothesis only given the evidence. (It is because there is no evidence that supports h, other than "e") Possibility that h is false only given e is the same as the possibility that the evidence does not support the hypothesis only given the evidence. (It is because there is no evidence that supports h). 20 Compu?ng 10

11 Possibility Distribu?on can be represented by a single value in [-1, 1]. Possibility distribu?on of hypothesis h given only e is represented by ( π (h!e), π (h!e) ), where max( π (h!e), π (h!e) ) =1.0 The possibility distribu?on could be represented by a single value gh(h!e) in [-1,1]. Then the possibility distribu?on is restored from gh(h!e) in [-1,1] using the following equa?ons. 21 Transforma?on between g h (h!e) and Possibility Distribu?ons g h (h!e) is regarded as a single value representa?on of a possibility distribu?on. 22 Compu?ng 11

12 Our Defini?on of Certainty Factors We define the CF by the next equa?on. 23 Aggrega?on of CFs (1) When two CFs x, y 0, it is supposed that the evidence ex and ey support the hypothesis h and In this case, x and y are transformed to the possibility distribu?ons. Then we calculate the possibility distribu?on of h given only ex and ey, using the proposi?ons shown before, and assuming condi?onal independency and non-interac?vity of causa?on events. π (h!e x,e y ) =1.0 π (h!e x,e y ) = min(1 x,1 y) Then, the above possibility distribu?on can be transformed back to a CF. 24 Compu?ng 12

13 Aggrega?on of CFs (2) When two CFs x, y 0, it is supposed that the evidence ex and ey support the O-hypothesis k and In this case, x and y are transformed to the following possibility distribu?ons. Then we calculate the possibility distribu?on of h given only ex and ey, using the proposi?ons assuming condi?onal independency and non-interac?vity of causa?on events. π (k!e x,e y ) =1.0 π (k!e x,e y ) = min(1+ x,1+ y) Then, the aggregated CF is obtained as follows; 25 Aggrega?on of CFs (3) When two CFs x > 0 > y, it is supposed that the evidence ex and ey support h and k, respec?vely and In this case, x and y are transformed to the following possibility distribu?ons. Then we calculate the possibility distribu?on of h given only ex and ey, using the proposi?ons assuming condi?onal independency and non-interac?vity of causa?on events. π (h k!e x,e y ) =1+ y Then, the aggregated CF is obtained as follows; π (h k!e x,e y ) =1 x 26 Compu?ng 13

14 Aggrega?on Func?ons of CFs By summing up the three cases before, we get the next aggrega?on func?on. The possibility distribu?on obtained in the case of x > 0 > y is not normal. (It is because x and y contradict each other) If we normalize the distribu?on, then transform it to CF, we get the next aggrega?on func?on. 27 Aggrega?on Func?ons of CFs (2) The standard opera?ons used for Possibility theory are Min and Max. Mathema?cally, it is possible to use other t-norm and t-conorm. If we use algebraic product and sum instead of Min and Max, we get the following aggrega?on func?ons. - If we do not normalize the possibility distribu?on in the case of x > 0 > y, - If we normalize the possibility distribu?on in the case of x > 0 > y, Note: This is completely the same as the MYCIN s aggrega?on func?on. The Certainty Factor model can be jus?fied in the framework of Possibility Theory. 28 Compu?ng 14

15 Mathema?cal Proper?es of the Four Aggrega?on Func?ons Agrega+on rule Commuta+vity Associa+vity Con+nuity Monotonicity MIN/MAX w/o normaliza?on Non-decreasing MIN/MAX w/ normaliza?on Non-decreasing Algebraic Prod/Sum w/o normaliza?on Increasing Algebraic Prod/Sum w/ normaliza?on Increasing 29 Numerical Example (1) Suppose we get five pieces of evidence for a hypothesis h. The CFs are given sequen?ally by 0.3, - 0.5, 0.8, 0.4, fmin : Min/Max operators, No normaliza?on fm-nor : Min/Max operators, with normaliza?on falg : Algebraic Product/Sum operators, No normaliza?on fa-nor : Algebraic Product/Sum operators, with normaliza?on 30 Compu?ng 15

16 Numerical Example (2) Aggrega?on results when the order of the sequence is changed. affected strongly by the recent informa?on Associa?vity 31 Effects of Repe??ve O-hypothesis with Low CF Suppose we have the hypothesis h with a high CF (0.9) at first, then the O-hypothesis k with a low CF (-0.1) is given repeatedly. - Hypothesis h : true news (CF = 0.9) - O-hypothesis k : fake news (CF = 0.1) Simula?on Result At first, CF = 0.9, then CF = -0.1 is repeated 20?mes. h -> k h -> k - Aggrega?ons without normaliza?on are sensi?ve to the repe??ve O-hypothesis, and the value of CF decreases rapidly. - But in the case of Min/Max opera?on, the result is bounded by the low CF (-0.1), while the case of algebraic opera?on accumulates the CF of the O-hypothesis. 32 Compu?ng 16

17 Effects of Repe??ve O-Hypothesis with Low CF(2) The case where the hypothesis h is given in the middle. The case where the hypothesis h is given in the end. h -> k h -> k - Aggrega?ons with algebraic opera?ons have an effect to accumulate the low CFs. => This effect is risky in cases where much fake informa?on is repeated. => On the other hand, it would be useful when we cannot get the certain direct evidence, and can get much indirect but reliable evidence with a low CF. 33 Lessons from Examples When much wrong evidence is contained, it is risky to use "algebraic opera?on" because of the accumula?on effects of CFs of wrong evidence. it is beoer use "normaliza?on", because decreasing effect of the CF by wrong evidence is small. In the situa?ons where there is much wrong evidence (e.g. noises), It is beoer to use Min/Max opera?on with normaliza?on. In cases where most of reliable evidence has low CFs and there is liole wrong evidence, aggrega?on with "algebraic opera?on" helps us. In the applica?ons where most of evidence is indirect with a low CF, but it is reliable, it is beoer to use algebraic opera?on with normaliza?on (e.g. clinical diagnosis). 34 Compu?ng 17

18 Posi?on of Epistemic Uncertainty in AI The goal: Inves?gate the Human Intelligence, and develop Human Like Machine Intelligence Ar?ficial Intelligence Epistemic AI The goal: analyze data, find paoerns, and develop machines to judge something using the paoerns. Symbolis?c AI (Tradi?onal AI) AI with Symbol processing symbols = concepts Computa?onal AI (Computa?onal Intelligence) AI with numerical processing Symbolis?c AI (Knowledge Discovery) Data Science/Engineering Computa?onal AI (Machine Learning) (Sta?s?cal AI) Problem-solving Heuris?c Search Deduc?ve reasoning Knowledge representa?on Connec?onism (Neural Nets) Fuzzy Logic Evolu?onal Computa?on Epistemic Uncertainty Aoribute Oriented Induc?on Rough Set Model Associa?on Learning Discriminant Analysis Support Vector Machine Decision Trees Ensemble Learning Bayesian Learning 35 Conclusion Epistemic Uncertainty was discussed. It is important even in the era of data science, as long as humans / robots have to make decisions based on much uncertain/vague informa?on. Theories of epistemic uncertainty should be able to deal with "ignorance" or "unknown situa?on" caused by lack of informa?on/knowledge. The CF model, which had been cri?cized for a long?me, was recalled, interpreted newly with Possibility theory, and the aggrega?on func?on was jus?fied theore?cally. Four simple aggrega?on func?ons of Certainty Factors were proposed, and the mathema?cal proper?es were discussed. 36 Compu?ng 18