Integration of Symbolic and Connectionist AI techniques

Transcription

1 Integration of Symbolic and Connectionist AI techniques Decision Support Systems for biochemical processes Davide Sottara Seminari III anno dottorato - XXII ciclo

2 Outline 1 PhD Curriculum 2 Introduction Case Study : Water Treatment State of the Art 3 Hybrid Architectures Proposed Architecture Hybrid Modules 4 Rule Engine Inference under Uncertainty Rule Language and Engine 5 Results and Future Works

3 Curriculum Tutor: Co-Tutor: Cooperations : Other: Prof. P. Mello Ing. L. Luccarini ENEA - ACS PROT IDR Water Resource Management Section (Jan 07 - Dec 09) University of Newcastle / JBoss (Feb 09 - Jun 09) Track Co-Chair at RULEML09 Rules and Uncertainty

4 Publications Journal Papers G. L. Bragadin, G. Colombini, L. Luccarini, M. Mancini, P. Mello, M. Montali, and D. Sottara. Formal verification of wastewater treatment processes using events detected from continuous signals by means of artificial neural networks. Case study: SBR plant. Environmental Modelling and Software (IF 2.659). Article in Press. P. Mello, M. Proctor, and D. Sottara. A configurable RETE-OO engine for reasoning with different types of imperfect information. IEEE Transactions on Knowledge and Data Engineering (TKDE) - Special Issue on Rule Representation, Interchange and Reasoning in Distributed, Heterogeneous Environments (IF 2.236). Article in Press.

5 Conference Acts I Sottara D., P.Mello, L.Luccarini, and G.Colombini. Controllo e gestione intelligente degli impianti di depurazione. In Europa del Recupero : le ricerche, le tecnologie, gli strumenti e i casi studio per una cultura della responsabilitã ambientale, pages , S.Arcangelo di Romagna (RN) ITA, 5-8 Novembre Maggioli Editore (ITALY). D.Sottara, L.Luccarini, and P.Mello. Strumenti di IA per il controllo e la diagnosi dei processi biologici negli impianti a fanghi attivi. In Europa del recupero : le ricerche, le tecnologie, gli strumenti e i casi studio per una cultura della responsabilitã ambientale, pages , S.Arcangelo di Romagna (RN) ITA, 5-8 Novembre Maggioli Editore (ITALY). L. Luccarini, P. Mello, D. Sottara, and A. Spagni. Artificial Intelligence based rules for event recognition and control applied to SBR systems. In Conference Proceedings of the 4th Sequencing Batch Reactor Conference, pages , ROMA ITA, 7-10 April, s.n. M. Nickles and D. Sottara. Approaches to Uncertain or Imprecise Rules - A survey. In G. Governatori, J. Hall, and A. Paschke, editors, Rule Interchange and Applications, International Symposium, RuleML 2009, Las Vegas, Nevada, USA, November 5-7, Proceedings, volume 5858 of Lecture Notes in Computer Science, pages Springer, D. Sottara and P. Mello. Modelling radial basis functions with rational logic rules. In E. Corchado, A. Abraham, and W. Pedrycz, editors, Hybrid Artificial Intelligence Systems, Third International Workshop, HAIS 2008, Burgos, Spain, September 24-26, Proceedings, volume 5271 of Lecture Notes in Computer Science, pages Springer, 2008.

6 Conference Acts II D. Sottara, L. Luccarini, P. Mello, S. Grilli, M. Mancini, and G.L. Bragadin. Tecniche di intelligenza artificiale per la gestione e il controllo di impianti di depurazione. caso di studio: SBR in scala pilota alimentato con refluo reale. In Luciano Morselli, editor, Ambiente: tecnologie, controlli e certificazioni per il recupero e la valorizzazione di materiali ed energie. ECOMONDO X Fiera Internazionale del Recupero di Materia ed Energia e dello Sviluppo Sostenibile. Rimini novembre 2006, volume 1, pages Maggioli Editore (ITALY), D. Sottara, L. Luccarini, and P. Mello. AI techniques for Waste Water Treatment Plant control. Case study: Denitrification in a pilot-scale SBR. In B. Apolloni, R. J. Howlett, and L. C. Jain, editors, Knowledge-Based Intelligent Information and Engineering Systems, 11th International Conference, KES 2007, XVII Italian Workshop on Neural Networks, Vietri sul Mare, Italy, September 12-14, Proceedings, Part I, volume 4692 of Lecture Notes in Computer Science, pages Springer, D. Sottara, P. Mello, and M. Proctor. Adding uncertainty to a RETE-OO inference engine. In N. Bassiliades, G. Governatori, and A. Paschke, editors, Rule Representation, Interchange and Reasoning on the Web, International Symposium, RuleML 2008, Orlando, FL, USA, October 30-31, Proceedings, volume 5321 of Lecture Notes in Computer Science, pages Springer, D. Sottara, G. Colombini, L. Luccarini, and P. Mello. A Pool of Experts to evaluate the evolution of biological processes in SBR plants. In E. Corchado, X. Wu, E. Oja, Á. Herrero, and B. Baruque, editors, Hybrid Artificial Intelligence Systems, 4th International Conference, HAIS 2009, Salamanca, Spain, June 10-12, Proceedings, volume 5572 of Lecture Notes in Computer Science, pages Springer, 2009.

7 Conference Acts III D. Sottara, L. Luccarini, G.L. Bragadin, M.L. Mancini, P. Mello, and M. Montali. Process quality assessment in automatic management of wastewater treatment plants using formal verification. In International Symposium on Sanitary and Environmental Engineering-SIDISA 08 -Proceedings, volume 1, pages 152/1 152/8, ROMA ITA, june ANDIS. D. Sottara, A. Manservisi, P. Mello, G. Colombini, and L. Luccarini. A CEP-based SOA for the management of wastewater treatment plants. In EESMS IEEE Workshop on Environmental, Energy, and Structural Monitoring Systems, Proceedings, pages 58 65, 25/09/ D. Sottara, P. Mello, L. Luccarini, G. Colombini, and A. Manservisi. Controllo intelligente in linea per una gestione efficiente e sostenibile degli impianti di trattamento reflui. Caso di studio: SBR in scala pilota. In Ecodesign per il pianeta: soluzioni per un ambiente pulito e per una nuova economia, pages , S.Arcangelo di Romagna (RN) ITA, Ottobre Maggioli Editore (ITALY). D. Sottara, P. Mello, and M. Proctor. Towards modelling defeasible reasoning with imperfection in production rule systems. In G. Governatori, J. Hall, and A. Paschke, editors, Rule Interchange and Applications, International Symposium, RuleML 2009, Las Vegas, Nevada, USA, November 5-7, Proceedings, volume 5858 of Lecture Notes in Computer Science, pages Springer, N. Wulff and D. Sottara. Fuzzy reasoning with a RETE-OO Rule Engine. In G. Governatori, J. Hall, and A. Paschke, editors, Rule Interchange and Applications, International Symposium, RuleML 2009, Las Vegas, Nevada, USA, November 5-7, Proceedings, volume 5858 of Lecture Notes in Computer Science, pages Springer, 2009.

8 Case Study : Water Treatment Outline 1 PhD Curriculum 2 Introduction Case Study : Water Treatment State of the Art 3 Hybrid Architectures Proposed Architecture Hybrid Modules 4 Rule Engine Inference under Uncertainty Rule Language and Engine 5 Results and Future Works

9 Case Study : Water Treatment Sequencing Batch Reactors Single treatment tank Cyclic process : Reactions sequential in time ph, redox potential orp, dissolved oxygen DO probes

10 Case Study : Water Treatment Intelligent Management of Complex Systems Control Process Optimization Greater efficiency Money/Energy savings Diagnosis Anomaly Prevention Fault Isolation Automatic Intervention Support Diagnosis Detection Probes Operator Intervention Prevention Reaction Actuators Plant

11 Case Study : Water Treatment Reaction Completion Change in signal trends are correlated to completed reactions Denitrification 2 Nitrification Draw Load Anox Aero Set Idle

12 State of the Art Outline 1 PhD Curriculum 2 Introduction Case Study : Water Treatment State of the Art 3 Hybrid Architectures Proposed Architecture Hybrid Modules 4 Rule Engine Inference under Uncertainty Rule Language and Engine 5 Results and Future Works

13 State of the Art A few considerations... The problem is complex

14 State of the Art A few considerations... The problem is complex Models are not applicable

15 State of the Art A few considerations... The problem is complex Models are not applicable Decision Support Systems are more suitable

16 State of the Art A few considerations... The problem is complex Models are not applicable Decision Support Systems are more suitable No single AI technology is optimal

17 State of the Art Integrated Intelligent Remote Control User Interface Integrated Remote Control System DSS Probes Database Plant Controller PID Actuators Combines different technologies Remote Access Diagnosis & Fault Detection Optimal Set-point Control Operator Support Benefits Full KB system Reactive / Proactive Limitations Monolithic architecture Coordination?

18 State of the Art Integrated Intelligent Remote Control User Interface Analysis Techniques Probes NDSS OD R F Database Plant Controller PID Actuators Different approaches Data Mining Neural Networks Fuzzy Logic Rule Based Systems Ontologies Benefits Full KB system Reactive / Proactive Limitations Monolithic architecture Coordination?

19 Proposed Architecture Outline 1 PhD Curriculum 2 Introduction Case Study : Water Treatment State of the Art 3 Hybrid Architectures Proposed Architecture Hybrid Modules 4 Rule Engine Inference under Uncertainty Rule Language and Engine 5 Results and Future Works

20 Proposed Architecture Complex Event-Driven SOAs CED-SOA A Service-Oriented Architecture for Complex Event Processing

21 Proposed Architecture Complex Event-Driven SOAs CED-SOA A Service-Oriented Architecture for Complex Event Processing Services may interact producing or consuming events Looser coupling Reactiveness

22 Proposed Architecture Complex Event-Driven SOAs CED-SOA A Service-Oriented Architecture for Complex Event Processing Services may interact producing or consuming events Looser coupling Reactiveness Services aggregate events Implementation is hidden

23 Proposed Architecture Complex Event-Driven SOAs CED-SOA A Service-Oriented Architecture for Complex Event Processing Services may interact producing or consuming events Looser coupling Reactiveness Services aggregate events Implementation is hidden The middleware delivers events Producers need not know Consumers, if any

24 Proposed Architecture Towards Complex Achitectures User Interface NDSS OD R F Probes Database Plant Ctrl + PID Actuators

25 Proposed Architecture Towards Complex Achitectures User Interface D N F R O Database Ctrl + PID DSS Probes Plant Actuators

26 Proposed Architecture Towards Complex Achitectures User Interface D N F R O Database Ctrl + PID DSS Probes Plant Actuators

27 Proposed Architecture Towards Complex Achitectures User Interface ORFN D Database Ctrl + PID DSS Probes Plant Actuators

28 Proposed Architecture Towards Complex Achitectures User Interface ORFN D Database Controller DSS Probes Plant Actuators

29 Proposed Architecture Towards Complex Achitectures User Interface ORFN D Acquisition Database Controller DSS Probes Plant Actuators

30 ORFN D Proposed Architecture Towards Complex Achitectures UI DW I/OStore Acquisition Controller Probes Plant Actuators

31 ORFN D Proposed Architecture Towards Complex Achitectures Security Admin Registry... UI DW I/OStore Acquisition Controller Probes Plant Actuators

32 ORFN D Proposed Architecture Towards Complex Achitectures Security Admin Registry... UI DW Scheduler I/OStore Acquisition Controller Probes Plant Actuators

33 ORFN D Proposed Architecture Towards Complex Achitectures Security Admin Registry... UI DW Scheduler Rule I/OStore Acquisition Controller Probes Plant Actuators

34 ORFN D Proposed Architecture Towards Complex Achitectures Security Admin Registry... UI DW Rule Router Scheduler I/OStore Acquisition Controller Probes Plant Actuators

35 Proposed Architecture Event-Processing Networks : (Loose) Interactions Chart Statistics Scheduler Predict Probes Denoise Analysis Policy Control Actuators Trace Storage Router

36 Proposed Architecture Events : A typical scenario Chart Statistics Scheduler Predict Probes Raw Denoise Analysis Policy Control Actuators Trace Storage Router

37 Proposed Architecture Events : A typical scenario Chart Statistics Scheduler Predict Probes Denoise Sample Analysis Policy Control Actuators Trace Storage Router

38 Proposed Architecture Events : A typical scenario Chart Statistics Scheduler Predict Estimate Probes Denoise AnalysisTrend Policy Control Actuators Trace Stage Storage Router

39 Proposed Architecture Events : A typical scenario Chart Statistics Scheduler Predict Probes Denoise Analysis Policy Switch Control Actuators Trace Storage Router

40 Proposed Architecture Events : A typical scenario Chart Statistics Scheduler Predict Probes Denoise Analysis Policy Control Actuators Trace Phase Storage Router

41 Proposed Architecture Events : A typical scenario Chart Statistics Scheduler Predict Switch Probes Denoise Analysis Policy Control Actuators Trace Storage Router

42 Hybrid Modules Outline 1 PhD Curriculum 2 Introduction Case Study : Water Treatment State of the Art 3 Hybrid Architectures Proposed Architecture Hybrid Modules 4 Rule Engine Inference under Uncertainty Rule Language and Engine 5 Results and Future Works

43 Hybrid Modules AI Modules Chart Statistics Scheduler P R Predict R FF SOM Probes Denoise Analysis Policy Control Actuators Num R Num R Trace R SOM Storage Router R

44 Hybrid Modules Hybrid Systems Combine benefits of Soft and Hard Computing Hard Computing (HCS) Encode Knowledge Self-Explanatory Reason Soft Computing (SCS) Learn Flexible Evaluate Complementary

45 Hybrid Modules Hybrid Systems Combine benefits of Soft and Hard Computing Hard Computing (HCS) Encode Knowledge Self-Explanatory Reason Soft Computing (SCS) Learn Flexible Evaluate Complementary Problem : Integration The output of SCS is Uncertain and unsuitable for HCS

46 Hybrid Modules An Ontology for Uncertainty (W3C) Uncertainty Nature Derivation Type Model Aleatory Episthemic Subjective Objective Inconsistency Vagueness Incompleteness FuzzySets Ambiguity Randomness Probability Belief RandomSets RoughSets more...

47 Hybrid Modules An Ontology for Uncertainty (W3C) Uncertainty Uncertainty / Confidence Factors Nature Derivation Type Model Aleatory Episthemic Subjective Objective Inconsistency Vagueness Incompleteness FuzzySets Ambiguity Randomness Probability Belief RandomSets RoughSets more...

48 Hybrid Modules An Ontology for Uncertainty (W3C) Uncertainty Uncertainty / Frequentist Probability Nature Derivation Type Model Aleatory Episthemic Subjective Objective Inconsistency Vagueness Incompleteness FuzzySets Ambiguity Randomness Probability Belief RandomSets RoughSets more...

49 Hybrid Modules An Ontology for Uncertainty (W3C) Uncertainty Uncertainty / Bayesian Probability Nature Derivation Type Model Aleatory Episthemic Subjective Objective Inconsistency Vagueness Incompleteness FuzzySets Ambiguity Randomness Probability Belief RandomSets RoughSets more...

50 Hybrid Modules An Ontology for Uncertainty (W3C) Uncertainty Vagueness / Fuzzy Logic Nature Derivation Type Model Aleatory Episthemic Subjective Objective Inconsistency Vagueness Incompleteness FuzzySets Ambiguity Randomness Probability Belief RandomSets RoughSets more...

51 Hybrid Modules Rule examples : Analysis r u l e "Step Up" // f u z z y r u l e when $ f : F e a t u r e s ( d e l t a T i s "short" and d e l t a Y i s "high" and l e f t D e r i s "flat" and cender i s " steeppositive" ) then TrendChange t c = new TrendChange ( $f, "step_up", d r o o l s. getconsequencedegree ( ) ) ; d e l i v e r E v e n t ( t c ) ; end

52 Hybrid Modules Rule examples : Prediction r u l e " Predict NO3 " // c a s c a d e d h y b r i d when $s : Sample ( $ i d : i d ) $n : NeuralNet ( $out : o u t p u t == " no3 " ) then i n s e r t ( new Value ( $id, $out, $n. e v a l ( $s ) ) ; end r u l e " Validate " // f u n c t i o n embedding h y b r i d when $s : Sample ( $ i d : i d ) $v : Value ( i d == $id, t y p e == " no3 " ) e x i s t s SOM Neuron ( t h i s s i m i l a r $s ) then i n s e r t ( new E s t i m a t e ( $s, $v, d r o o l s. getconsequencedegree ( ) ) ; end

53 Hybrid Modules Rule examples : Policy r u l e " EoD " // time aware r u l e w i t h f u z z y >p r o b a b i l i t y mapping i m p l i c a t i o k i n d= f u z 2 p r o b, p r i o r = i d e n t i t y ] when $ f : CurrPhase ( name == " anox " ) k i n d= Luk ] // c o n f i g a t t r i b u t e s $m : TrendChange ( s i g n a l == "ph", t y p e == " max " ) and $k : TrendChange ( s i g n a l == " orp ", t y p e == " knee_down " ) and TrendChange ( t h i s == $m, t h i s o v e r l a p s $k ) then EndOfReact eod = new EndOfReact ( " denitro ", d r o o l s. getconsequencedegree ( ) ; d e l i v e r E v e n t ( eod ) ; end

54 Hybrid Modules Rule examples : Policy II r u l e "Switch" // i n j e c t i n g r u l e when $ f : CurrPhase ( name == "anox" ) $eod : EndOfReact ( r e a c t i o n == "denitro" ) then Switch sw = new Switch (+1); // next phase i n j e c t (new Tuple ( sw ), "holds" ) ; i n s e r t ( sw ) ; end

55 Hybrid Modules Rule examples : Policy III r u l e "Safe_Switch" // multi premise, g r a d u a l r u l e when $s : Switch ( t h i s holds ) kind= prod ] / p ( S ) r ( S ) > p( S ) r ( S ) / i m p l i e s ( Switch ( t h i s c r i s p ] $s, t h i s neg holds and t h i s neg cost "falsen" ) Switch ( t h i s c r i s p ] $s, t h i s holds and t h i s neg cost "falsep" ) ) then s c h e d u l e ( $s, TMAX (1 d r o o l s. getconsequencedegree ( ) ) ) ; end

56 Outline 1 PhD Curriculum 2 Introduction Case Study : Water Treatment State of the Art 3 Hybrid Architectures Proposed Architecture Hybrid Modules 4 Rule Engine Inference under Uncertainty Rule Language and Engine 5 Results and Future Works

57 DROOLS JBoss Drools Business Rule Management System Production Rules : RETE-based Open Source Modular Expert : Object-Oriented Rule engine Flow : Support for Workflows Fusion : Support for Events Guvnor : Remote rule Repository

58 Inference under Uncertainty Outline 1 PhD Curriculum 2 Introduction Case Study : Water Treatment State of the Art 3 Hybrid Architectures Proposed Architecture Hybrid Modules 4 Rule Engine Inference under Uncertainty Rule Language and Engine 5 Results and Future Works

59 Inference under Uncertainty Generalized Inference P(x),P(X ) C(Y ) C(y) Classic Modus Ponens Premise and Implication entail Consequence

60 Inference under Uncertainty Generalized Inference Φ(...,A j (x)/ε j,... ),P(X ) C(Y ) C(y) Premise Atomic constraints are evaluated General, pluggable Evaluators A Degree is returned

61 Inference under Uncertainty Generalized Inference Φ(...,A j (x)/ε j,... )/ε P,P(X ) C(Y ) C(y) Premise Atomic constraints are evaluated General, pluggable Evaluators A Degree is returned Premise Atoms are aggregated in formulas using generalized logic Connectives evaluated by Operators

62 Inference under Uncertainty Generalized Inference Implication Implication has a Degree often given a priori P(x)/ε P, (X,Y ) /ε C(y)

63 Inference under Uncertainty Generalized Inference P(x)/ε P, (X,Y ) /ε C(y)/ε C Implication Implication has a Degree often given a priori Modus Ponens MP computes the Degree of the Consequence

64 Inference under Uncertainty Generalized Inference P 1, 1 C 1 /ε C1,..., P n, n Cn/ε C n C(y)/ε C Merging multiple sources Multiple premises for the same conclusion Solve conflicts Handle missing values

65 Rule Language and Engine Outline 1 PhD Curriculum 2 Introduction Case Study : Water Treatment State of the Art 3 Hybrid Architectures Proposed Architecture Hybrid Modules 4 Rule Engine Inference under Uncertainty Rule Language and Engine 5 Results and Future Works

66 Rule Language and Engine Language extensions Custom Evaluators Before : limited support for boolean functions After : integration with external modules Adapter interfaces Degrees carry more information Formulas Before : conjunction, quantifiers, NaF After : support for all standard connectives Configuration Attributes Before : parameters passed to custom evaluators only After : granular configuration Compile-time : choose implementation Run-time : configure propagation behaviour

67 Rule Language and Engine Language extensions : Example r u l e "Rule" // custom : i m p l i c a t i o n s and MP i m p l i c a t i o degree = 0.75 ] d e d u c t i o kind= min ] when $o1 : Type ( $f1 : f i e l d 1 / custom : e x t e r n a l e v a l u a t o r / i d= i 1, kind= e x t e r n a l, params =... ] "val" ) kind= max ] // custom : o p e r a t o r s $o2 : AnotherType ( f i e l d 3 == 0 ˆˆ // custom : o p e r a t o r s f i e l d 3 c r i s p ] $f1 ) // custom : b e h a v i o u r then / consequence d e g r e e /... = d r o o l s. getconsequencedegree ( ) ;

68 Rule Language and Engine Engine Extension Global additions Evaluations and Degrees Centralized Factory Builds and converts degrees and operators Improved RETE Network Additional Nodes Enabled Node Operator Nodes Including Implication and Modus Ponens Augmented Alpha and Beta nodes

69 Rule Language and Engine Extended Engine : Example # # #4 3 #11 3 #3 1 #10 1 #2 field1 == val #9$f1 2 #1 Type $o1 # #0 this enabled #7 field3 == 0 # #5 #6 this AnotherType enabled $o2 #8 field3 == $f1 #12 2 #13 1 #14 0 #15 #Rule 2

70 Outline 1 PhD Curriculum 2 Introduction Case Study : Water Treatment State of the Art 3 Hybrid Architectures Proposed Architecture Hybrid Modules 4 Rule Engine Inference under Uncertainty Rule Language and Engine 5 Results and Future Works

71 Results and Future Works Results Design and development of a configurable RETE engine Added support for different non-boolean logics Development of strongly coupled hybrid systems Technology transfer : application of modern technologies to WWTPs

72 Results and Future Works Results Design and development of a configurable RETE engine Added support for different non-boolean logics Development of strongly coupled hybrid systems Technology transfer : application of modern technologies to WWTPs Future Developments Release the engine as an official module ( Drools Chance ) Integration of Rule-Based Systems and Ontologies Application to different domains