EXTENDED FUZZY COGNITIVE MAPS

Size: px
Start display at page:

Download "EXTENDED FUZZY COGNITIVE MAPS"

Transcription

1 EXTENDED FUZZY COGNITIVE MAPS Masafumi Hagiwara Department of Psychology Stanford University Stanford, CA 930 U.S.A. Abstract Fuzzy Cognitive Maps (FCMs) have been proposed to represent causal reasoning by using numeric processing. They graphically represent uncertain causal reasoning. In the resonant states, there emerges a limit cycle or a hidden pattern, which is a FCM inference. However, there are some shortcomings concerned with knowledge representation in the conventional FCMs. In this paper, we propose Extended Fuzzy Cognitive Maps (E-FCMs) to represent causal relationships more naturally. The features of the E-FCMs are: there are nonlinear membership functions, conditional weights, and time delay weights. Computer simulation results indicate the effectiveness of the E-FCMs. 1. Introduction Knowledge representation and inference have been very important and difficult problems in Artificial Intelligence (AI). Traditional AI systems such as expert systems can store and process knowledge; generally they are based on symbolic, not numeric. In symbolic processing, language strings represent thought or short-term memory, rules and relations between language strings correspond to long-term memory, and programming corresponds learning. In traditional AI systems, it is easy to represent structured knowledge as rules. However, symbolic processing is not good at dealing with mathematical analysis in the traditional senses of engineering and the physical sciences; it is hard to directly apply the tools of numerical mathematics. Neural networks have a numerical framework with theorems and algorithms. They have demonstrated their superior performance in some applications (Rumelhart, McClelland, et al., 1986, Gorman & Sejnowski, 1988,Waibel, Hanazawa, Hinton, Shikano & Lang, 1989). However there are still some drawbacks: they cannot directly encode structured knowledge, and generally learning requires many cycles. Fuzzy Cognitive Maps (FCMs) have been proposed to represent causal reasoning (Kosko, 198, Taber & Siegel, 198, Zhang & Chen, 1988). FCMs are useful not only to symbolic representation of semantic networks, but also to distributed representation of cognitive processes. An addition, it is quite appealing that a biologically observed excitatory (positive) and inhibitory (negative) associations between neurons are represented in this logical architecture (Zhang and Chen, 1988). The features of the FCMs are: 1) They store domain knowledge in the nodes and the directional connections. ) They graphically represent uncertain causal reasoning. 3) Their matrix representation allow causal inferences to be made as feedback associative memory recollections. Since FCMs are dynamical systems, their resonant states are limit cycles. The limit cycle or hidden pattern is a FCM inference (Kosko, 1991). However, there are three important drawbacks in the conventional FCMs: 1) Connections in FCMs are just numeric ones: relationship of two events should be linear. ) Lack of a concept of time; practically each causal has different time delay. 3) They cannot deal with co-occurrence of multiple causes such as expressed by "and" conditions /9 $ IEEE 9

2 In this paper, we propose Extended Fuzzy Cognitive Maps (E-FCMs) to overcome the above drawbacks. The features of the E-FCM are as follows: 1) Weights having nonlinear membership functions. ) Conditional weights. 3) Time delay weights. In, conventional FCMs are briefly explained. In 3, E-FCMs are proposed. And then computer simulation results are shown in.. Conventional Fuzzy Cognitive Maps (FCM) In this section, the structure and behavior of Fuzzy Cognitive Maps (FCMs) are briefly explained. Generally FCMs are representated by fuzzy signed digraph with feedback. A FCM graphically represents uncertain causal reasoning. FCM causal connection weights are numbers in [- 1, 11 to allow degrees of causality to be represented. Fig. 1 shows an example of a simple FCM for director of public health (Zhang & Chen, 1988). The FCM has the following causal connection matrix E. c, c c3 c, cs c, c r Fig. 1 An example of a simple Fuz~y Cognitive Map. 96

3 In addition, any set of FCMs can be naturally combined. Each expert can draw a different size of FCM causal concepts. There is no restriction on the number of experts or on the number of concepts. To combine arbitrary FCM connection matrices E,, E,, * * e, E,, first, these matrices are augmented. Each augmented matrix Fi has n rows and n columns, where n is the total number of distinct concept used by the experts. The combined matrix can be expressed as (Kosko, 198), R F =x W, r=l F, where wi is a nonnegative credibility weight of each expert. 3. Proposed Extended Fuzzy Cognitive Maps (E-FCM) In this section, first,three drawbacks of the conventional FCMs are mentioned, and then proposed Extended Fuzzy Cognitive Maps (E-FCM) are explained. As for the conventional FCMs, these are three important drawbacks. 1) Connections in FCMs are just numeric ones; relationship of two events should be linear such as shown in Fig.(a). ) Lack of a concept of time; practically each causal has different time delay. 3) They cannot deal with co-occurrence of multiple causes such as (Gupta, 1991, Kaufmann & Gupta, 1983), if X,, and X, -1 and... and Xn, then Y. (3 1 As for l), according to the connection wi.<, If (the number of people in a city increases) then (modernization of the city advances) with the proportion of 0.6. () However, it is not true in a sense "with the proportion of 0.6'; the most populous city is not necessarily the most modernized. In general the relationship (0.6 in this case) is not a linear but a nonlinear, such as shown in Fig.(b)..- 3 cd e.,u.- n,k - d c.c 8 - C d % E, " 9

4 As for ), it is natural that the time delay in the connection ws6 [if (sanitation facilities are improved) then (the number of diseases per lo00 residents decreases)] is longer than that in the connections w,[if (migration in a city increases) then (the number of people in a city)j. As for 3), if the government of the country controls migration, connection w3 changes; w3 should be conditional such as: if (modernization of a city advances) and (the government restricts migration) () then (migration into the country increases) with the proportion x. Considering the above discussions, we propose Extended Fuzzy Cognitive Maps (E-FCMs). In E-FCMs, total input to node Cj at time t can be expressed as, where, C,(t) is a causal concept at time t, w,(.) is a weight function from concept C,(t) to concept C,(r), and defuvlj is a time delay from causal concept C,(t) to concept C,(t). Similar to the conventional FCMs, output of each causal node CJ(t) is a nonlinear function that transforms the path-weighted activation flowing into it into a value in 10, 11. This nonlinear function is in general a bounded monotonous increasing transformation, such as a sigmoid or S-shaped functions. Fig.3 An e\ample of it simple Extended Furq Cognitile Map. 98

5 . Computer simulations Here, computer simulation results are shown to demonstrate the superiority of E-FCMs. The same example as in Fig. 1 (Zhang and Chen, 1988) was used for the conventional FCM. As for the proposed E-FCMs, the map shown in Fig.3 was used. We modified the conventional FCM just parts for simplicity, which are shown by dotted circles in Fig.3. a) Nonlinear weights: C, -->C, (~13) and C, -->C, (w p) Conditional weight: C,, C, -->C, (w3,=0./s). If (the government controls migration, C,) then {if (migration into the country increases) then (modernization of the city advances) with proportion 0./ (not 0.)) () y) Time delays: there is one time delay in the weights C, -->C, (ws6). There are, of course, many parts to be changed in Fig. I, however, we changed just the above parts to make clear the effects. Table I shows each node activation as a function of time for the case of the conventional FCM. Table shows that for the case of the proposed E-FCMs. Table (a) is a result using only a), Table (b) is a result using a) and p), and Table (c) is a result using a), p). and y). It can be observed many things from these tables. For example, the modernization (C,) in the conventional FCM is saturated from time, on the other hand as for the E-FCMs, when there are two nonlinear weights (wi3 and wl,), it is saturated from time 8 (Table (a)). When there are nonlinear weights and a conditional weight, it is from 10 (Table (b)). And the case of Table (c), it is from 11. Another conspicuous thing is about "bacteria per area (C,)". For the conventional FCM, the value of C, converges to 0.08, not 0. However, for the proposed E-FCMs, they converge to 0. From above results and discussion, it is clear that nonlinear weights, conditional weights, and time delay in weights greatly influence the behavior of FCM. -. Tinic in a tit\ into a cit). Lation C CS C6 c Garbage Sanitation n of peop~c per Bacteria per area facilities 100oresideiice per area I I.ooo I.ooo , ooo ooo I.000 I.ooo h 8 9 IO Tablc 1 hch ntdc actit ation as ii function of time for the case of the con\entional FCM 99

6 Cl c C3 C Cs Ch c C8 'I ime # of people Rligration hlodemi- Garbage Sanitation # of people per Bacteria ('ontrol bq in a cit) into a city Lation pe r area racilities IOOOresidence per area government 0 I r,. I ime. 0 I # of people hligration Modemi- Garbage Sanitation #of people per Bacteria ('ontrol by in a city into a citj lation per area facilities 1000residcnce per area government : OOO X into a city dation pe r area facilities IOOOresidenee per area government I so I

7 . Conclusions In this paper, we have proposed Extended Fuzzy Cognitive Maps (E-FCMs) to represent causal relationship more naturally. The features of the E-FCMs are as follows: 1) Weights having nonlinear membership functions. ) Conditional weights. 3) Time delay weights. Computer simulation results have demonstrated the effectiveness of the proposed E-FCMs. The proposed methods will greatly improve the behavior of FCM. The example used in this paper was a simple one, however, the proposed E-FCMs are useful to many fields such as cognitive science, social science, computer vision, and so on. Acknowledgement The auther wish to thank Prof. David E. Rumelhart for his valuable comments to improve this paper. REFERENCES Gorman, R.P. & Sejnowski, T.J. (1988). Analysis of hidden units in a layered network trained to classify sonar targets. Neurul Networks, 1, -89. Gupta, M.M., and Qi, J. (1991). On fuzzy neuron models. Proceeding of Internutionul Joint Conference on Neural Networks,, Kaufmann, A., and Gupta, M.M. (1988). Fuzzy mathematical models in engineering and management science. North-Holland. Kosko, B. ( 198). Adaptive inference in fuzzy knowledge networks. Proceeding of the First International Conference on Neural Networks,, Kosko, B. (1991). Neural networks and fuzzy system. Prentice Hall. Rumelhart, D.E., McClelland, J.L. & the PDP Research Group (1986). Parallel Distributed Processing, MIT Press. Taber, W.R., and Siegel, M.A. (198). Estimation of expert weights using fuzzy cognitive maps. Proceeding of the First Internutionul Conference on Neural h'etworkb,, Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., and Lang, K., (1989). Phoneme recognition using time-delay neural networks. IEEE Truns. on Acoustics, Speech, and Signal Processing, 3, Zhang, W., and Chen, S. (1988). A logical architecture for cognitive maps. Proceeding of the - nd International Conference on Neural Networks, 1,

Temporal Backpropagation for FIR Neural Networks

Temporal Backpropagation for FIR Neural Networks Temporal Backpropagation for FIR Neural Networks Eric A. Wan Stanford University Department of Electrical Engineering, Stanford, CA 94305-4055 Abstract The traditional feedforward neural network is a static

More information

Artificial Neural Network

Artificial Neural Network Artificial Neural Network Contents 2 What is ANN? Biological Neuron Structure of Neuron Types of Neuron Models of Neuron Analogy with human NN Perceptron OCR Multilayer Neural Network Back propagation

More information

Knowledge Extraction from DBNs for Images

Knowledge Extraction from DBNs for Images Knowledge Extraction from DBNs for Images Son N. Tran and Artur d Avila Garcez Department of Computer Science City University London Contents 1 Introduction 2 Knowledge Extraction from DBNs 3 Experimental

More information

EEE 241: Linear Systems

EEE 241: Linear Systems EEE 4: Linear Systems Summary # 3: Introduction to artificial neural networks DISTRIBUTED REPRESENTATION An ANN consists of simple processing units communicating with each other. The basic elements of

More information

A FUZZY NEURAL NETWORK MODEL FOR FORECASTING STOCK PRICE

A FUZZY NEURAL NETWORK MODEL FOR FORECASTING STOCK PRICE A FUZZY NEURAL NETWORK MODEL FOR FORECASTING STOCK PRICE Li Sheng Institute of intelligent information engineering Zheiang University Hangzhou, 3007, P. R. China ABSTRACT In this paper, a neural network-driven

More information

Keywords- Source coding, Huffman encoding, Artificial neural network, Multilayer perceptron, Backpropagation algorithm

Keywords- Source coding, Huffman encoding, Artificial neural network, Multilayer perceptron, Backpropagation algorithm Volume 4, Issue 5, May 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Huffman Encoding

More information

Fuzzy Cognitive Maps Learning through Swarm Intelligence

Fuzzy Cognitive Maps Learning through Swarm Intelligence Fuzzy Cognitive Maps Learning through Swarm Intelligence E.I. Papageorgiou,3, K.E. Parsopoulos 2,3, P.P. Groumpos,3, and M.N. Vrahatis 2,3 Department of Electrical and Computer Engineering, University

More information

DEEP LEARNING AND NEURAL NETWORKS: BACKGROUND AND HISTORY

DEEP LEARNING AND NEURAL NETWORKS: BACKGROUND AND HISTORY DEEP LEARNING AND NEURAL NETWORKS: BACKGROUND AND HISTORY 1 On-line Resources http://neuralnetworksanddeeplearning.com/index.html Online book by Michael Nielsen http://matlabtricks.com/post-5/3x3-convolution-kernelswith-online-demo

More information

fraction dt (0 < dt 1) from its present value to the goal net value: Net y (s) = Net y (s-1) + dt (GoalNet y (s) - Net y (s-1)) (2)

fraction dt (0 < dt 1) from its present value to the goal net value: Net y (s) = Net y (s-1) + dt (GoalNet y (s) - Net y (s-1)) (2) The Robustness of Relaxation Rates in Constraint Satisfaction Networks D. Randall Wilson Dan Ventura Brian Moncur fonix corporation WilsonR@fonix.com Tony R. Martinez Computer Science Department Brigham

More information

EE-559 Deep learning Recurrent Neural Networks

EE-559 Deep learning Recurrent Neural Networks EE-559 Deep learning 11.1. Recurrent Neural Networks François Fleuret https://fleuret.org/ee559/ Sun Feb 24 20:33:31 UTC 2019 Inference from sequences François Fleuret EE-559 Deep learning / 11.1. Recurrent

More information

COMP-4360 Machine Learning Neural Networks

COMP-4360 Machine Learning Neural Networks COMP-4360 Machine Learning Neural Networks Jacky Baltes Autonomous Agents Lab University of Manitoba Winnipeg, Canada R3T 2N2 Email: jacky@cs.umanitoba.ca WWW: http://www.cs.umanitoba.ca/~jacky http://aalab.cs.umanitoba.ca

More information

Application of Artificial Neural Networks in Evaluation and Identification of Electrical Loss in Transformers According to the Energy Consumption

Application of Artificial Neural Networks in Evaluation and Identification of Electrical Loss in Transformers According to the Energy Consumption Application of Artificial Neural Networks in Evaluation and Identification of Electrical Loss in Transformers According to the Energy Consumption ANDRÉ NUNES DE SOUZA, JOSÉ ALFREDO C. ULSON, IVAN NUNES

More information

CS 4700: Foundations of Artificial Intelligence

CS 4700: Foundations of Artificial Intelligence CS 4700: Foundations of Artificial Intelligence Prof. Bart Selman selman@cs.cornell.edu Machine Learning: Neural Networks R&N 18.7 Intro & perceptron learning 1 2 Neuron: How the brain works # neurons

More information

Introduction Biologically Motivated Crude Model Backpropagation

Introduction Biologically Motivated Crude Model Backpropagation Introduction Biologically Motivated Crude Model Backpropagation 1 McCulloch-Pitts Neurons In 1943 Warren S. McCulloch, a neuroscientist, and Walter Pitts, a logician, published A logical calculus of the

More information

EE04 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb 23, Sasidharan Sreedharan

EE04 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb 23, Sasidharan Sreedharan EE04 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb 23, 2012 Sasidharan Sreedharan www.sasidharan.webs.com 3/1/2012 1 Syllabus Artificial Intelligence Systems- Neural Networks, fuzzy logic,

More information

Address for Correspondence

Address for Correspondence Research Article APPLICATION OF ARTIFICIAL NEURAL NETWORK FOR INTERFERENCE STUDIES OF LOW-RISE BUILDINGS 1 Narayan K*, 2 Gairola A Address for Correspondence 1 Associate Professor, Department of Civil

More information

Deep Belief Networks are compact universal approximators

Deep Belief Networks are compact universal approximators 1 Deep Belief Networks are compact universal approximators Nicolas Le Roux 1, Yoshua Bengio 2 1 Microsoft Research Cambridge 2 University of Montreal Keywords: Deep Belief Networks, Universal Approximation

More information

INTRODUCTION TO NEURAL NETWORKS

INTRODUCTION TO NEURAL NETWORKS INTRODUCTION TO NEURAL NETWORKS R. Beale & T.Jackson: Neural Computing, an Introduction. Adam Hilger Ed., Bristol, Philadelphia and New York, 990. THE STRUCTURE OF THE BRAIN The brain consists of about

More information

t Correspondence should be addressed to the author at the Department of Psychology, Stanford University, Stanford, California, 94305, USA.

t Correspondence should be addressed to the author at the Department of Psychology, Stanford University, Stanford, California, 94305, USA. 310 PROBABILISTIC CHARACTERIZATION OF NEURAL MODEL COMPUTATIONS Richard M. Golden t University of Pittsburgh, Pittsburgh, Pa. 15260 ABSTRACT Information retrieval in a neural network is viewed as a procedure

More information

The Robustness of Relaxation Rates in Constraint Satisfaction Networks

The Robustness of Relaxation Rates in Constraint Satisfaction Networks Brigham Young University BYU ScholarsArchive All Faculty Publications 1999-07-16 The Robustness of Relaxation Rates in Constraint Satisfaction Networks Tony R. Martinez martinez@cs.byu.edu Dan A. Ventura

More information

Reification of Boolean Logic

Reification of Boolean Logic 526 U1180 neural networks 1 Chapter 1 Reification of Boolean Logic The modern era of neural networks began with the pioneer work of McCulloch and Pitts (1943). McCulloch was a psychiatrist and neuroanatomist;

More information

Neural Networks: Introduction

Neural Networks: Introduction Neural Networks: Introduction Machine Learning Fall 2017 Based on slides and material from Geoffrey Hinton, Richard Socher, Dan Roth, Yoav Goldberg, Shai Shalev-Shwartz and Shai Ben-David, and others 1

More information

A SEASONAL FUZZY TIME SERIES FORECASTING METHOD BASED ON GUSTAFSON-KESSEL FUZZY CLUSTERING *

A SEASONAL FUZZY TIME SERIES FORECASTING METHOD BASED ON GUSTAFSON-KESSEL FUZZY CLUSTERING * No.2, Vol.1, Winter 2012 2012 Published by JSES. A SEASONAL FUZZY TIME SERIES FORECASTING METHOD BASED ON GUSTAFSON-KESSEL * Faruk ALPASLAN a, Ozge CAGCAG b Abstract Fuzzy time series forecasting methods

More information

Information Sciences

Information Sciences Information Sciences 178 (2008) 3880 3894 Contents lists available at ScienceDirect Information Sciences journal homepage: www.elsevier.com/locate/ins Comparing the inference capabilities of binary, trivalent

More information

Artificial Neural Network and Fuzzy Logic

Artificial Neural Network and Fuzzy Logic Artificial Neural Network and Fuzzy Logic 1 Syllabus 2 Syllabus 3 Books 1. Artificial Neural Networks by B. Yagnanarayan, PHI - (Cover Topologies part of unit 1 and All part of Unit 2) 2. Neural Networks

More information

Part 8: Neural Networks

Part 8: Neural Networks METU Informatics Institute Min720 Pattern Classification ith Bio-Medical Applications Part 8: Neural Netors - INTRODUCTION: BIOLOGICAL VS. ARTIFICIAL Biological Neural Netors A Neuron: - A nerve cell as

More information

CS 343: Artificial Intelligence

CS 343: Artificial Intelligence CS 343: Artificial Intelligence Particle Filters and Applications of HMMs Prof. Scott Niekum The University of Texas at Austin [These slides based on those of Dan Klein and Pieter Abbeel for CS188 Intro

More information

Extending the Associative Rule Chaining Architecture for Multiple Arity Rules

Extending the Associative Rule Chaining Architecture for Multiple Arity Rules Extending the Associative Rule Chaining Architecture for Multiple Arity Rules Nathan Burles, James Austin, and Simon O Keefe Advanced Computer Architectures Group Department of Computer Science University

More information

Competitive Learning for Deep Temporal Networks

Competitive Learning for Deep Temporal Networks Competitive Learning for Deep Temporal Networks Robert Gens Computer Science and Engineering University of Washington Seattle, WA 98195 rcg@cs.washington.edu Pedro Domingos Computer Science and Engineering

More information

International Journal of Advanced Research in Computer Science and Software Engineering

International Journal of Advanced Research in Computer Science and Software Engineering Volume 3, Issue 4, April 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Application of

More information

Multilayer Perceptron Tutorial

Multilayer Perceptron Tutorial Multilayer Perceptron Tutorial Leonardo Noriega School of Computing Staffordshire University Beaconside Staffordshire ST18 0DG email: l.a.noriega@staffs.ac.uk November 17, 2005 1 Introduction to Neural

More information

A Logarithmic Neural Network Architecture for Unbounded Non-Linear Function Approximation

A Logarithmic Neural Network Architecture for Unbounded Non-Linear Function Approximation 1 Introduction A Logarithmic Neural Network Architecture for Unbounded Non-Linear Function Approximation J Wesley Hines Nuclear Engineering Department The University of Tennessee Knoxville, Tennessee,

More information

Pattern Recognition Prof. P. S. Sastry Department of Electronics and Communication Engineering Indian Institute of Science, Bangalore

Pattern Recognition Prof. P. S. Sastry Department of Electronics and Communication Engineering Indian Institute of Science, Bangalore Pattern Recognition Prof. P. S. Sastry Department of Electronics and Communication Engineering Indian Institute of Science, Bangalore Lecture - 27 Multilayer Feedforward Neural networks with Sigmoidal

More information

Radial-Basis Function Networks

Radial-Basis Function Networks Radial-Basis Function etworks A function is radial () if its output depends on (is a nonincreasing function of) the distance of the input from a given stored vector. s represent local receptors, as illustrated

More information

Computational Intelligence Lecture 3: Simple Neural Networks for Pattern Classification

Computational Intelligence Lecture 3: Simple Neural Networks for Pattern Classification Computational Intelligence Lecture 3: Simple Neural Networks for Pattern Classification Farzaneh Abdollahi Department of Electrical Engineering Amirkabir University of Technology Fall 2011 arzaneh Abdollahi

More information

Human-Oriented Robotics. Temporal Reasoning. Kai Arras Social Robotics Lab, University of Freiburg

Human-Oriented Robotics. Temporal Reasoning. Kai Arras Social Robotics Lab, University of Freiburg Temporal Reasoning Kai Arras, University of Freiburg 1 Temporal Reasoning Contents Introduction Temporal Reasoning Hidden Markov Models Linear Dynamical Systems (LDS) Kalman Filter 2 Temporal Reasoning

More information

Fibring Neural Networks

Fibring Neural Networks Fibring Neural Networks Artur S d Avila Garcez δ and Dov M Gabbay γ δ Dept of Computing, City University London, ECV 0H, UK aag@soicityacuk γ Dept of Computer Science, King s College London, WC2R 2LS,

More information

An artificial neural networks (ANNs) model is a functional abstraction of the

An artificial neural networks (ANNs) model is a functional abstraction of the CHAPER 3 3. Introduction An artificial neural networs (ANNs) model is a functional abstraction of the biological neural structures of the central nervous system. hey are composed of many simple and highly

More information

Associative Memory : Soft Computing Course Lecture 21 24, notes, slides RC Chakraborty, Aug.

Associative Memory : Soft Computing Course Lecture 21 24, notes, slides   RC Chakraborty,  Aug. Associative Memory : Soft Computing Course Lecture 21 24, notes, slides www.myreaders.info/, RC Chakraborty, e-mail rcchak@gmail.com, Aug. 10, 2010 http://www.myreaders.info/html/soft_computing.html www.myreaders.info

More information

Neural Networks Introduction

Neural Networks Introduction Neural Networks Introduction H.A Talebi Farzaneh Abdollahi Department of Electrical Engineering Amirkabir University of Technology Winter 2011 H. A. Talebi, Farzaneh Abdollahi Neural Networks 1/22 Biological

More information

Learning and Memory in Neural Networks

Learning and Memory in Neural Networks Learning and Memory in Neural Networks Guy Billings, Neuroinformatics Doctoral Training Centre, The School of Informatics, The University of Edinburgh, UK. Neural networks consist of computational units

More information

COMPARING PERFORMANCE OF NEURAL NETWORKS RECOGNIZING MACHINE GENERATED CHARACTERS

COMPARING PERFORMANCE OF NEURAL NETWORKS RECOGNIZING MACHINE GENERATED CHARACTERS Proceedings of the First Southern Symposium on Computing The University of Southern Mississippi, December 4-5, 1998 COMPARING PERFORMANCE OF NEURAL NETWORKS RECOGNIZING MACHINE GENERATED CHARACTERS SEAN

More information

Deep Learning: Self-Taught Learning and Deep vs. Shallow Architectures. Lecture 04

Deep Learning: Self-Taught Learning and Deep vs. Shallow Architectures. Lecture 04 Deep Learning: Self-Taught Learning and Deep vs. Shallow Architectures Lecture 04 Razvan C. Bunescu School of Electrical Engineering and Computer Science bunescu@ohio.edu Self-Taught Learning 1. Learn

More information

On Computational Limitations of Neural Network Architectures

On Computational Limitations of Neural Network Architectures On Computational Limitations of Neural Network Architectures Achim Hoffmann + 1 In short A powerful method for analyzing the computational abilities of neural networks based on algorithmic information

More information

Equivalence of Backpropagation and Contrastive Hebbian Learning in a Layered Network

Equivalence of Backpropagation and Contrastive Hebbian Learning in a Layered Network LETTER Communicated by Geoffrey Hinton Equivalence of Backpropagation and Contrastive Hebbian Learning in a Layered Network Xiaohui Xie xhx@ai.mit.edu Department of Brain and Cognitive Sciences, Massachusetts

More information

COMP304 Introduction to Neural Networks based on slides by:

COMP304 Introduction to Neural Networks based on slides by: COMP34 Introduction to Neural Networks based on slides by: Christian Borgelt http://www.borgelt.net/ Christian Borgelt Introduction to Neural Networks Motivation: Why (Artificial) Neural Networks? (Neuro-)Biology

More information

Artificial Neural Networks. Historical description

Artificial Neural Networks. Historical description Artificial Neural Networks Historical description Victor G. Lopez 1 / 23 Artificial Neural Networks (ANN) An artificial neural network is a computational model that attempts to emulate the functions of

More information

CS 4700: Foundations of Artificial Intelligence

CS 4700: Foundations of Artificial Intelligence CS 4700: Foundations of Artificial Intelligence Prof. Bart Selman selman@cs.cornell.edu Machine Learning: Neural Networks R&N 18.7 Intro & perceptron learning 1 2 Neuron: How the brain works # neurons

More information

Supervised (BPL) verses Hybrid (RBF) Learning. By: Shahed Shahir

Supervised (BPL) verses Hybrid (RBF) Learning. By: Shahed Shahir Supervised (BPL) verses Hybrid (RBF) Learning By: Shahed Shahir 1 Outline I. Introduction II. Supervised Learning III. Hybrid Learning IV. BPL Verses RBF V. Supervised verses Hybrid learning VI. Conclusion

More information

A DYNAMIC FUZZY-COGNITIVE-MAP APPROACH BASED ON RANDOM NEURAL NETWORKS

A DYNAMIC FUZZY-COGNITIVE-MAP APPROACH BASED ON RANDOM NEURAL NETWORKS International Journal of Computational Cognition (http://www.yangsky.com/yangijcc.htm) Volume 1, Number 4, Pages 91 107, December 2003 Publisher Item Identifier S 1542-5908(03)10405-8/$20.00 Article electronically

More information

(Feed-Forward) Neural Networks Dr. Hajira Jabeen, Prof. Jens Lehmann

(Feed-Forward) Neural Networks Dr. Hajira Jabeen, Prof. Jens Lehmann (Feed-Forward) Neural Networks 2016-12-06 Dr. Hajira Jabeen, Prof. Jens Lehmann Outline In the previous lectures we have learned about tensors and factorization methods. RESCAL is a bilinear model for

More information

CS 343: Artificial Intelligence

CS 343: Artificial Intelligence CS 343: Artificial Intelligence Particle Filters and Applications of HMMs Prof. Scott Niekum The University of Texas at Austin [These slides based on those of Dan Klein and Pieter Abbeel for CS188 Intro

More information

Artificial Intelligence

Artificial Intelligence Artificial Intelligence Jeff Clune Assistant Professor Evolving Artificial Intelligence Laboratory Announcements Be making progress on your projects! Three Types of Learning Unsupervised Supervised Reinforcement

More information

Multilayer Feedforward Networks. Berlin Chen, 2002

Multilayer Feedforward Networks. Berlin Chen, 2002 Multilayer Feedforard Netors Berlin Chen, 00 Introduction The single-layer perceptron classifiers discussed previously can only deal ith linearly separable sets of patterns The multilayer netors to be

More information

Adaptive Inverse Control based on Linear and Nonlinear Adaptive Filtering

Adaptive Inverse Control based on Linear and Nonlinear Adaptive Filtering Adaptive Inverse Control based on Linear and Nonlinear Adaptive Filtering Bernard Widrow and Gregory L. Plett Department of Electrical Engineering, Stanford University, Stanford, CA 94305-9510 Abstract

More information

AI Programming CS F-20 Neural Networks

AI Programming CS F-20 Neural Networks AI Programming CS662-2008F-20 Neural Networks David Galles Department of Computer Science University of San Francisco 20-0: Symbolic AI Most of this class has been focused on Symbolic AI Focus or symbols

More information

Neural Networks and Machine Learning research at the Laboratory of Computer and Information Science, Helsinki University of Technology

Neural Networks and Machine Learning research at the Laboratory of Computer and Information Science, Helsinki University of Technology Neural Networks and Machine Learning research at the Laboratory of Computer and Information Science, Helsinki University of Technology Erkki Oja Department of Computer Science Aalto University, Finland

More information

SIMULATION OF FREEZING AND FROZEN SOIL BEHAVIOURS USING A RADIAL BASIS FUNCTION NEURAL NETWORK

SIMULATION OF FREEZING AND FROZEN SOIL BEHAVIOURS USING A RADIAL BASIS FUNCTION NEURAL NETWORK SIMULATION OF FREEZING AND FROZEN SOIL BEHAVIOURS USING A RADIAL BASIS FUNCTION NEURAL NETWORK Z.X. Zhang 1, R.L. Kushwaha 2 Department of Agricultural and Bioresource Engineering University of Saskatchewan,

More information

COMP9444 Neural Networks and Deep Learning 2. Perceptrons. COMP9444 c Alan Blair, 2017

COMP9444 Neural Networks and Deep Learning 2. Perceptrons. COMP9444 c Alan Blair, 2017 COMP9444 Neural Networks and Deep Learning 2. Perceptrons COMP9444 17s2 Perceptrons 1 Outline Neurons Biological and Artificial Perceptron Learning Linear Separability Multi-Layer Networks COMP9444 17s2

More information

Bidirectional Representation and Backpropagation Learning

Bidirectional Representation and Backpropagation Learning Int'l Conf on Advances in Big Data Analytics ABDA'6 3 Bidirectional Representation and Bacpropagation Learning Olaoluwa Adigun and Bart Koso Department of Electrical Engineering Signal and Image Processing

More information

A gradient descent rule for spiking neurons emitting multiple spikes

A gradient descent rule for spiking neurons emitting multiple spikes A gradient descent rule for spiking neurons emitting multiple spikes Olaf Booij a, Hieu tat Nguyen a a Intelligent Sensory Information Systems, University of Amsterdam, Faculty of Science, Kruislaan 403,

More information

A Study on the Causes for Failures in Mathematics by Engineering Students Using CFRM Model

A Study on the Causes for Failures in Mathematics by Engineering Students Using CFRM Model , July 2-4, 2014, London, U.K. A Study on the Causes for Failures in Mathematics by Engineering Students Using CFRM Model A. Praveen Prakash, J. Esther Jerlin, J. Bennilo Fernandes Abstract: The quality

More information

Using Neural Networks for Identification and Control of Systems

Using Neural Networks for Identification and Control of Systems Using Neural Networks for Identification and Control of Systems Jhonatam Cordeiro Department of Industrial and Systems Engineering North Carolina A&T State University, Greensboro, NC 27411 jcrodrig@aggies.ncat.edu

More information

Recurrent and Recursive Networks

Recurrent and Recursive Networks Neural Networks with Applications to Vision and Language Recurrent and Recursive Networks Marco Kuhlmann Introduction Applications of sequence modelling Map unsegmented connected handwriting to strings.

More information

Modeling Zero Energy Building with a Three- Level Fuzzy Cognitive Map

Modeling Zero Energy Building with a Three- Level Fuzzy Cognitive Map Modeling Zero Energy Building with a Three- Level Fuzzy Cognitive Map Eleni S. Vergini 1, Theodora-Eleni Ch. Kostoula 2, P. P. Groumpos 3 Abstract The concept of Zero Energy Buildings (ZEB) is briefly

More information

SGD and Deep Learning

SGD and Deep Learning SGD and Deep Learning Subgradients Lets make the gradient cheating more formal. Recall that the gradient is the slope of the tangent. f(w 1 )+rf(w 1 ) (w w 1 ) Non differentiable case? w 1 Subgradients

More information

Artificial Neural Networks The Introduction

Artificial Neural Networks The Introduction Artificial Neural Networks The Introduction 01001110 01100101 01110101 01110010 01101111 01101110 01101111 01110110 01100001 00100000 01110011 01101011 01110101 01110000 01101001 01101110 01100001 00100000

More information

CS 354R: Computer Game Technology

CS 354R: Computer Game Technology CS 354R: Computer Game Technology AI Fuzzy Logic and Neural Nets Fall 2017 Fuzzy Logic Philosophical approach Decisions based on degree of truth Is not a method for reasoning under uncertainty that s probability

More information

Artificial Neural Network Simulation of Battery Performance

Artificial Neural Network Simulation of Battery Performance Artificial work Simulation of Battery Performance C.C. O Gorman, D. Ingersoll, R.G. Jungst and T.L. Paez Sandia National Laboratories PO Box 58 Albuquerque, NM 8785 Abstract Although they appear deceptively

More information

ARTIFICIAL NEURAL NETWORK PART I HANIEH BORHANAZAD

ARTIFICIAL NEURAL NETWORK PART I HANIEH BORHANAZAD ARTIFICIAL NEURAL NETWORK PART I HANIEH BORHANAZAD WHAT IS A NEURAL NETWORK? The simplest definition of a neural network, more properly referred to as an 'artificial' neural network (ANN), is provided

More information

CHAPTER 3. Pattern Association. Neural Networks

CHAPTER 3. Pattern Association. Neural Networks CHAPTER 3 Pattern Association Neural Networks Pattern Association learning is the process of forming associations between related patterns. The patterns we associate together may be of the same type or

More information

A New Method to Forecast Enrollments Using Fuzzy Time Series

A New Method to Forecast Enrollments Using Fuzzy Time Series International Journal of Applied Science and Engineering 2004. 2, 3: 234-244 A New Method to Forecast Enrollments Using Fuzzy Time Series Shyi-Ming Chen a and Chia-Ching Hsu b a Department of Computer

More information

Sections 18.6 and 18.7 Analysis of Artificial Neural Networks

Sections 18.6 and 18.7 Analysis of Artificial Neural Networks Sections 18.6 and 18.7 Analysis of Artificial Neural Networks CS4811 - Artificial Intelligence Nilufer Onder Department of Computer Science Michigan Technological University Outline Univariate regression

More information

Equivalence in Knowledge Representation: Automata, Recurrent Neural Networks, and Dynamical Fuzzy Systems

Equivalence in Knowledge Representation: Automata, Recurrent Neural Networks, and Dynamical Fuzzy Systems Equivalence in Knowledge Representation: Automata, Recurrent Neural Networks, and Dynamical Fuzzy Systems C. LEE GILES, FELLOW, IEEE, CHRISTIAN W. OMLIN, AND KARVEL K. THORNBER Invited Paper Neurofuzzy

More information

Analysis of Multilayer Neural Network Modeling and Long Short-Term Memory

Analysis of Multilayer Neural Network Modeling and Long Short-Term Memory Analysis of Multilayer Neural Network Modeling and Long Short-Term Memory Danilo López, Nelson Vera, Luis Pedraza International Science Index, Mathematical and Computational Sciences waset.org/publication/10006216

More information

Lecture 4: Feed Forward Neural Networks

Lecture 4: Feed Forward Neural Networks Lecture 4: Feed Forward Neural Networks Dr. Roman V Belavkin Middlesex University BIS4435 Biological neurons and the brain A Model of A Single Neuron Neurons as data-driven models Neural Networks Training

More information

Water Quality Management using a Fuzzy Inference System

Water Quality Management using a Fuzzy Inference System Water Quality Management using a Fuzzy Inference System Kumaraswamy Ponnambalam and Seyed Jamshid Mousavi A fuzzy inference system (FIS) is presented for the optimal operation of a reservoir system with

More information

Automatic Differentiation Equipped Variable Elimination for Sensitivity Analysis on Probabilistic Inference Queries

Automatic Differentiation Equipped Variable Elimination for Sensitivity Analysis on Probabilistic Inference Queries Automatic Differentiation Equipped Variable Elimination for Sensitivity Analysis on Probabilistic Inference Queries Anonymous Author(s) Affiliation Address email Abstract 1 2 3 4 5 6 7 8 9 10 11 12 Probabilistic

More information

Machine Learning. Neural Networks. (slides from Domingos, Pardo, others)

Machine Learning. Neural Networks. (slides from Domingos, Pardo, others) Machine Learning Neural Networks (slides from Domingos, Pardo, others) For this week, Reading Chapter 4: Neural Networks (Mitchell, 1997) See Canvas For subsequent weeks: Scaling Learning Algorithms toward

More information

Density Propagation for Continuous Temporal Chains Generative and Discriminative Models

Density Propagation for Continuous Temporal Chains Generative and Discriminative Models $ Technical Report, University of Toronto, CSRG-501, October 2004 Density Propagation for Continuous Temporal Chains Generative and Discriminative Models Cristian Sminchisescu and Allan Jepson Department

More information

System Identification for the Hodgkin-Huxley Model using Artificial Neural Networks

System Identification for the Hodgkin-Huxley Model using Artificial Neural Networks Proceedings of International Joint Conference on Neural Networks, Orlando, Florida, USA, August 12-17, 2007 System Identification for the Hodgkin-Huxley Model using Artificial Neural Networks Manish Saggar,

More information

CMSC 421: Neural Computation. Applications of Neural Networks

CMSC 421: Neural Computation. Applications of Neural Networks CMSC 42: Neural Computation definition synonyms neural networks artificial neural networks neural modeling connectionist models parallel distributed processing AI perspective Applications of Neural Networks

More information

Artificial Neural Networks. Q550: Models in Cognitive Science Lecture 5

Artificial Neural Networks. Q550: Models in Cognitive Science Lecture 5 Artificial Neural Networks Q550: Models in Cognitive Science Lecture 5 "Intelligence is 10 million rules." --Doug Lenat The human brain has about 100 billion neurons. With an estimated average of one thousand

More information

Bayesian probability theory and generative models

Bayesian probability theory and generative models Bayesian probability theory and generative models Bruno A. Olshausen November 8, 2006 Abstract Bayesian probability theory provides a mathematical framework for peforming inference, or reasoning, using

More information

Land Use Changes Modeling Based on Different Approaches: Fuzzy Cognitive Maps, Cellular Automata and Neural Networks

Land Use Changes Modeling Based on Different Approaches: Fuzzy Cognitive Maps, Cellular Automata and Neural Networks Land Use Changes Modeling Based on Different Approaches: Fuzzy Cognitive Maps, Cellular Automata and Neural Networks MARIA MOISE Computer Science for Business Management Faculty Romanian American University

More information

Distance in Matrices and Their Applications to Fuzzy Models and Neutrosophic Models. W. B. Vasantha Kandasamy Florentin Smarandache Ilanthenral K

Distance in Matrices and Their Applications to Fuzzy Models and Neutrosophic Models. W. B. Vasantha Kandasamy Florentin Smarandache Ilanthenral K Distance in Matrices and Their Applications to Fuzzy Models and Neutrosophic Models W. B. Vasantha Kandasamy Florentin Smarandache Ilanthenral K 2014 This book can be ordered from: EuropaNova ASBL Clos

More information

Unit 8: Introduction to neural networks. Perceptrons

Unit 8: Introduction to neural networks. Perceptrons Unit 8: Introduction to neural networks. Perceptrons D. Balbontín Noval F. J. Martín Mateos J. L. Ruiz Reina A. Riscos Núñez Departamento de Ciencias de la Computación e Inteligencia Artificial Universidad

More information

Neural Networks with Applications to Vision and Language. Feedforward Networks. Marco Kuhlmann

Neural Networks with Applications to Vision and Language. Feedforward Networks. Marco Kuhlmann Neural Networks with Applications to Vision and Language Feedforward Networks Marco Kuhlmann Feedforward networks Linear separability x 2 x 2 0 1 0 1 0 0 x 1 1 0 x 1 linearly separable not linearly separable

More information

ARTIFICIAL INTELLIGENCE. Artificial Neural Networks

ARTIFICIAL INTELLIGENCE. Artificial Neural Networks INFOB2KI 2017-2018 Utrecht University The Netherlands ARTIFICIAL INTELLIGENCE Artificial Neural Networks Lecturer: Silja Renooij These slides are part of the INFOB2KI Course Notes available from www.cs.uu.nl/docs/vakken/b2ki/schema.html

More information

Machine Learning. Neural Networks

Machine Learning. Neural Networks Machine Learning Neural Networks Bryan Pardo, Northwestern University, Machine Learning EECS 349 Fall 2007 Biological Analogy Bryan Pardo, Northwestern University, Machine Learning EECS 349 Fall 2007 THE

More information

Reinforcement Learning, Neural Networks and PI Control Applied to a Heating Coil

Reinforcement Learning, Neural Networks and PI Control Applied to a Heating Coil Reinforcement Learning, Neural Networks and PI Control Applied to a Heating Coil Charles W. Anderson 1, Douglas C. Hittle 2, Alon D. Katz 2, and R. Matt Kretchmar 1 1 Department of Computer Science Colorado

More information

CS 5522: Artificial Intelligence II

CS 5522: Artificial Intelligence II CS 5522: Artificial Intelligence II Particle Filters and Applications of HMMs Instructor: Wei Xu Ohio State University [These slides were adapted from CS188 Intro to AI at UC Berkeley.] Recap: Reasoning

More information

Speech and Language Processing

Speech and Language Processing Speech and Language Processing Lecture 5 Neural network based acoustic and language models Information and Communications Engineering Course Takahiro Shinoaki 08//6 Lecture Plan (Shinoaki s part) I gives

More information

CSCI 315: Artificial Intelligence through Deep Learning

CSCI 315: Artificial Intelligence through Deep Learning CSCI 315: Artificial Intelligence through Deep Learning W&L Winter Term 2017 Prof. Levy Recurrent Neural Networks (Chapter 7) Recall our first-week discussion... How do we know stuff? (MIT Press 1996)

More information

A Sliding Mode Controller Using Neural Networks for Robot Manipulator

A Sliding Mode Controller Using Neural Networks for Robot Manipulator ESANN'4 proceedings - European Symposium on Artificial Neural Networks Bruges (Belgium), 8-3 April 4, d-side publi., ISBN -9337-4-8, pp. 93-98 A Sliding Mode Controller Using Neural Networks for Robot

More information

Neural Networks. Chapter 18, Section 7. TB Artificial Intelligence. Slides from AIMA 1/ 21

Neural Networks. Chapter 18, Section 7. TB Artificial Intelligence. Slides from AIMA   1/ 21 Neural Networks Chapter 8, Section 7 TB Artificial Intelligence Slides from AIMA http://aima.cs.berkeley.edu / 2 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural

More information

ECE 471/571 - Lecture 17. Types of NN. History. Back Propagation. Recurrent (feedback during operation) Feedforward

ECE 471/571 - Lecture 17. Types of NN. History. Back Propagation. Recurrent (feedback during operation) Feedforward ECE 47/57 - Lecture 7 Back Propagation Types of NN Recurrent (feedback during operation) n Hopfield n Kohonen n Associative memory Feedforward n No feedback during operation or testing (only during determination

More information

Bayesian Networks: Construction, Inference, Learning and Causal Interpretation. Volker Tresp Summer 2016

Bayesian Networks: Construction, Inference, Learning and Causal Interpretation. Volker Tresp Summer 2016 Bayesian Networks: Construction, Inference, Learning and Causal Interpretation Volker Tresp Summer 2016 1 Introduction So far we were mostly concerned with supervised learning: we predicted one or several

More information

Data Mining Part 5. Prediction

Data Mining Part 5. Prediction Data Mining Part 5. Prediction 5.5. Spring 2010 Instructor: Dr. Masoud Yaghini Outline How the Brain Works Artificial Neural Networks Simple Computing Elements Feed-Forward Networks Perceptrons (Single-layer,

More information

CS 5522: Artificial Intelligence II

CS 5522: Artificial Intelligence II CS 5522: Artificial Intelligence II Particle Filters and Applications of HMMs Instructor: Alan Ritter Ohio State University [These slides were adapted from CS188 Intro to AI at UC Berkeley. All materials

More information