Umetne Nevronske Mreže UNM (ANN)
|
|
- Allan Parker
- 6 years ago
- Views:
Transcription
1 Umetne Nevronske Mreže UNM (ANN) Glede na način učenja se ločujejo na Nenadzorovane (1) Nadzorovane () (1) Kohonenove nevronske mreže () Nevronske mreže z vzratnim širjenjem napake (error back propagation NN) (..1) Protitočne nevronske mreže (counterpropagation NN) () RBF nevronske mreže (radial basis function) 1
2 Začetki 194: McCulloch in Pitts, "A Logical Calculus of Ideas Immanent in Nervous Activity" Leta1949: Donald Hebb, The Organization of Behavior, predlaga Hebb-ovo pravilo (ojačitev povezave med sosednjimi vzbujenimi nevroni, ta aktivnost je osnova za učenje in memoriranje). Leta 196: Frank Rosenblatt, knjiga Principles of Neurodynamics z uporabo McCulloch-Pitts-ovega nevrona in Hebbovega pravila razvije prvi perceptron. Leta1969: Minsky in Papert, knjiga Perceptroni utemeljitev, zakaj imajo preceptroni teoretične omejitve pri reševanju nelinearnih problemov. Leta 198: John Hopfield, vpelje nelinearnost v osnovno funkcijo nevrona. S tem odpravi pomanjkljivosti in odpre novo obdobje raziskav umetnih nevronskih mrež.
3 XOR Input x 1 Input x Output
4 Tuevo Kohonen (rojen 194), Finski akademik. Selforganizing maps Kohonen, Leta1960: T. Kohonen vpelje nov koncept v neural computing asociativni spomin, optimalno asociativno mapiranje, self-organizing feature maps (SOMs). Kohonen, T. and Honkela, T. (007). "Kohonen network Leta 1986: Rumelhart s sodelavci objavili nov algoritem za tako nevronske mreže s vzvratnim širjenjem napake (error back-propagation) 4
5 Nature, 5-56 (09 October 1986); doi:10.108/5a0 Learning representations by back-propagating errors DAVID E. RUMELHART *, GEOFFREY E. HINTON & RONALD J. WILLIAMS * * Institute for Cognitive Science, C-015, University of California, San Diego, La Jolla, California 909, USA Department of Computer Science, Carnegie-Mellon University, Pittsburgh, Philadelphia 151, USA To whom correspondence should be addressed. We describe a new learning procedure, back-propagation, for networks of neurone-like units. The procedure repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector. As a result of the weight adjustments, internal 'hidden' units which are not part of the input or output come to represent important features of the task domain, and the regularities in the task are captured by the interactions of these units. The ability to create useful new features distinguishes back-propagation from earlier, simpler methods such as the perceptron-convergence procedure 1. References 1. Rosenblatt, F. Principles of Neurodynamics (Spartan, Washington, DC, 1961).. Minsky, M. L. & Papert, S. Perceptrons (MIT, Cambridge, 1969).. Le Cun, Y. Proc. Cognitiva 85, (1985). 4. Rumelhart, D. E., Hinton, G. E. & Williams, R. J. in Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol. 1: Foundations (eds Rumelhart, D. E. & McClelland, J. L.) 18 6 (MIT, Cambridge, 1986). 5
6 Predstavitev nevronov Vhodni signali X x 1 x x j x m Sinapse ali uteži w j Telo nevrona Akson Uteži w j Telo nevrona x 1 x x j x m Izhodni signal y Izhodni signal y
7 Izhodni signali nevronov Error Back Propagation nevronske mreže (perceptron) Net j = S i=1,k w ji x i a ( Net q ) y 1 1 e Y signal out y = f (Net ) y = f (Net ) a y = f (Net ) a Net 0 Net 0 Net q q q a) b) c) 7
8 Različne oblike EBP nevronskih mrež x 1 x x 1 x Net j = S i=1,k w ji x i w 11 w 1 w 1 w 1 w w w 11 w 1 w 1 w 1 w w b w 1b w w b y 1 y y y 1 y y x 1 x Bias w h 11 w h 1 w h 1 w h w h b Skriti nivo w h ji (6 uteži) Izhodni nivo w o ji (9 uteži) w o 11 w o 1 w h 1b w o 1 w o 1 w o w o 1b w o b w o w o b y 1 y y 8
9 w l ji hidden j Popravljanje uteži v nevronski mreži z vzvratnim širjenjem napake out n l j r ( k k 1 l 1 j output w w T o k p o p r a v k o v output kj )y l,prejšnji ji hidden j ( 1 vhodna plast: ni popravkov. popravki uteži v prvi skriti plasti. popravki uteži v drugi skriti plasti 1. popravki uteži v izhodni plasti signal out y hidden j ) x s1 x s bias signal 1 e Neaktivna vhodna plast: razporeditev spremenljivk x si 1. transformacija iz x si v out sj 1. transformacija iz out sj 1 v out sk. transformacija iz out sk v y sj 1 a ( Net q ) Net j = S i=1,k w ji x i T o k p o d a t k o v output j output output (t y )y ( 1 j j j y output j ) y s1 y s y s 9
10 V izračunu poravka vsake uteži upoštevamo podatke iz treh plasti nevronov i-ti nevron v skriti-1 plasti da izhodni signal enak y i hidden-1 w l ji out l j l 1 j w l,prejšnji ji j-ti nevron v skriti plasti da signal enak y hidden i w ji hidden Skrita plast w 1j output w j output w kj output w rj output Izhodna plast Izhodni nevroni, ki dajo signal y k output 1 1 output output k k output r r output y 1 output y output y k output y r output hidden j n r ( k k 1 output w output kj )y hidden j ( 1 y hidden j ) 10
11 Kohonenova nevronska mreža Vhodni objekt X s x 1 x x x 4 x 5 x 6 x 7 x 8 Kohonenova mreža Nivoji uteži Nivo uteži št. Nivo uteži št. 6 Posamezna utež 11
12 1 x 1 x x x 4 y 1 y y y 5 y n
13 x 1 x x x 4 x 1 x x x 4 y y 1 y y j y n y n-1 y n y 1 y a c c d b d Heksagonalna postavitev nevronov 1
14 A A A A,C B A A A,C C B A C C C B C C C B C C C B B C C C,B B B B B C C B B B B , , , , , , , ,
15 Mapa uteži spremenljivke x koncentracija pigmenta A Zgornja mapa s celicami, ki vsebujejo informacijo o kvaliteti barve (oznake A, B, in C) Vhodni objekt X s (receptura za barvo) predstavljen s tremi spremenljivkami Solvent x 1 Pigment x Binder x
16 Protitočna (counterpropagation) nevronska mreža Vhodna Kohonenova plast Vhodni objekt: X s x s1 x s x s x s4 x s5 x s6 x s7 x s8 Vzbujeni nevron W e Položaj vzbujenega nvrona W e se preslika v izhodno Grosbergovo plast Izhodna ali Grossbergova plast nevronov t s1 t s t s t s4 Potem, ko preslikamo položaj W e odgovor T s :vstopi v izhodno Grossbergovo plast 16
17 Radial Basis Function nevronske mreže Input: x 1 x Vhodni nivo 5 RBF nodes Bias w 1 w w w 4 w 5 w bias Output: y 1 17
18 RMSE RMSE training RMSE test Best model Best RMSE for test set Epohe (v tisočih) 18
19 Concentration of SO [mg/m ] Input Hidden Output (=input) 100 x 1 x x X j 400 x m 17 input variables are the same 17 output variables as targets Output value of the hidden node 1 19
20 KOHONENOVE IN PROTITOČNE (COUNTERPROPAGATION) NEVRONSKE MREŽE Študijska literatura 1. J. Zupan, J. Gasteiger, Neural Networks in Chemistry and Drug Design, Wiley-VCH, Weinheim, J. Zupan, Računalniške metode za kemike, DZS, Ljubljana, 199. Teach/Me software, Hans Lohninger, Springer Verlag Berlin T. Kohonen, Self-Organizing Maps, Springer-Verlag Berlin R. Hecht-Nielsen, "Counterpropagation networks, Applied Optics, 6, , R. Hecht-Nielsen, "Applications of counterpropagation networks, Neural Networks, 1, 11-19, R. Hecht-Nielsen, Neurocomputing, Reading, MA: Addison-Wesley, J. Zupan, M. Novič, I. Ruisánchez. Kohonen and counterpropagation artificial neural networks in analytical chemistry : tutorial. Chemometr. intell. lab. syst., 1997, vol. 8, str
21 Vsebina: Osnovna struktura umetnih nevronskih mrež Kohonenove umetne nevronske mreže (K-UNM) Protitočne (Counterpropagation) umetne nevronske mreže -D preslikava (mapiranje) s Kohonenovo UNM Klasifikacija Modeliranje Primeri uporabe Mapiranje IR spektrov Modeli za napovedovanje toksičnosti (log(1/ld50) za Tetrahymena pyroformis), preko 00 hidroksisubstituiranih aromatskih spojin 1
22 Umetne nevronske mreže (UNM) kot črna skrinja pri odločitvah vhodni signali izhodni signali strategije učenja nenadzorovano nadzorovano
23 UNM kot črna skrinja pri odločitvah Razpoznavanje vzorcev, določitev lastnosti, kontrola procesov... Vhodni signal UNM + Net k Vhodni signal: X=(x 1,x,...x i,...x m ) UNM postane uporabna šele potem, ko jo naučimo naših podatkov. Izhodni signal Izhodni signal: Y=(y 1,y,...y j,...y n )
24 Nenadzorovano učenje Razmerje med objektom in tarčo ni vnaprej definirano. Objekt X=(x 1,x,...x i,...x m ) Tarča Y=(ni poznana) 4
25 Nenadzorovano učenje Primer: Izvor italjanskih oljčnih olj Objekt: X=(x 1,x,...x i,...x 8 ) 57 vzorcev Tarča: Y (ni poznana) x 1...x 8 : % maščobnih kislin: (1)palmitinska, ()palmitoleinska, ()stearinska, (4)oleinska, (5)linoleinska, (6)arašidna (eikosanoična), (7)linolenska, (8)eikosenoična 5
26 Razred Pokrajina 1 North Apulia 5 Calabria 56 South Apulia 06 4 Sicily 6 5 Inner Sardinia 65 6 Coastal Sardinia 7 East Liguria 50 8 West Liguria 50 9 Umbria 51 S 57 Število vzorcev J. Zupan, M. Novic, X. Li, J. Gasteiger, Classification of Multicomponent Analytical Data of Olive Oils Using Different Neural Networks, Anal. Chim. Acta, 9, (1994),
27 Nadzorovano učenje Razmerje med objektom in tarčo je poznano vnaprej, kar pomeni, da za poljuben objekt natančno vemo, katera tarča mu pripada oziroma jo želimo doseči. Objekt Tarča X=(x 1,x,...x i,...x m ) Y=(y 1,y,...y j,...y n ) 7
28 Nadzorovano učenje Primer: Klasifikacija točk glede na kvadrante v kartezičnem koordinatnem sistemu Objekt: X=(x 1,x,...x i,...x m ) Tarča: Y=(y 1,y,...y j,...y n ) m= n=1 n= X=( 1, 1) Y=1 Y=(1,0,0,0) X=(-1, 1) Y= Y=(0,1,0,0) X=( -1,-1) Y= Y=(0,0,1,0) X=(1,-1) Y=4 Y=(0,0,0,1) 8
29 Nadzorovano učenje Primer Y= x Y= 1 Y= Y= 4 x 1 X=(x 1,x ) Y=(y 1,y,y,y 4 ) n=1 n= Y=1 Y=(1,0,0,0) Y= Y=(0,1,0,0) Y= Y=(0,0,1,0) Y=4 Y=(0,0,0,1) 9
30 Osnove Kohonenovih umetnih nevronskih mrež - organizacija nevronov - 1-dimenzionalna (v vrsti) - -dimenzionalna (v ravnini) - analogija Kohonenove UNM z možgani - učna strategija - zmagovalec dobi vse - parameteri hitrosti učenja - velikost in obseg popravkov uteži - krčenje območja sosedov - razpoznavanje objektov iz učnega niza - napoved neznanih objektov - vrhnja plast VP ( top-layer ) z oznakami - prazni prostori v VP - klastri v VP - konflikti v VP - nivoji uteži 0
31 Nevron je definiran kot niz uteži W j j-ti nevron; w ji i-ta utež j-tega nevrona W j [w ji i=1 m ] w j1 w j w j w j4 w j5 w j6 1
32 Organizacija nevronov W j j=1,n w ji, i=1,m n: število nevronov m: dimenzija nevronov Dimenzija nevronov m mora biti enaka dimenziji reprezentacije objektov (številu neodvisnih spremenljivk)!
33 1-dimenzionalna organizacija nevronov (v vrsti) X [x,x,x ] 1 m= W W W W W W n=6 W 1 W W W 1 W W W 4 W W W W W 5 W 6 W 4 W 5 W W 6 W
34 -dimenzionalna organizacija nevronov (v ravnini) X [x,x,x ] 1 j =1,6 x n = 6 x Število nevronov je n = n x n y = 4 (6x4) j =1,4 y n = 4 y W j x,j y,1 W j,j, x y W j,j, x y W j,j x y 4
35 Kohonenove nevronske mreže lahko uporabljamo za nelinearne projekcije več-dimenzionalnih objektov v dvo-dimenzionalen prostor ravnino (mapiranje) "homunculus" V možganih so področja nevronov, ki kontrolirajo gibanje motoriko posameznih delov človeškega telesa, organizirana eno zraven drugega, kot na navideznem zemljevidu človeškega telesa. Vendar velikost posameznih področij ne ustreza velikosi organa, katerega gibanje kontrolira. Velikost področja je sorazmerna pomembnosti in kompleksnosti njegovih aktivnosti. 5
36 "homunculus" 6
37 Strategija učenja - zmagovalec dobi vse" Winner = W win min (ε j ) x i w ji ε j ej å m ( xi w ji) i 1 objekt - komponente vektorja X j ti nevron komponente vektorja W (uteži) razlika med objektom in j tim nevronom 7
38 Analogija med biološkim in simuliranim nevronom: W =W win j x j y Signal vzbudi en sam nevron v celem šopu realnih, bioloških nevronov. j = y j =5 x V računalniški simulaciji je vzbujeni (zmagovalni) nevron W win določen s koordinato svojega položaja v UNM W win =W j xjy 8
39 POPRAVLJANJE: zmagovalni" nevron + okoliški nevroni Parameter hitrosti učenja: η(t,r(t))) t: čas učenja r(t): okolica (sosedni nevroni) w novi ji w stari ji ( i 1, m j 1, n s 1, p x si w stari ji ) 9
40 Proces, v katerem popravimo zmagovalni nevron in ustrezne okoliške nevrone glede na en sam (s ti ) objekt, imenujemo ena iteracija učnega procesa. Potem ko skozi mrežo pošljemo enega za drugim vse objekte (p) in naredio ustrezne popravke uteži, pravimo, da smo zaključili eno učno epoho. p iteracij = 1 epoha 40
41 Ena epoha učenja Kohonenove mreže e X - s W stari sosed s=1 X s W stari sosed e ji W novi sosed NET w i n n e r n e i g h b o u r s s=s+1 Ponavljaj do s=p obj p : število objektov v učnem nizu w novi ji w stari ji ( x w si 41 ) stari ji
42 Velikost in obseg popravkov: Krčenje obsega sosedov max r Flat function Triangular function Mexican hat function r c Chef hat function 4
43 4
44 Razpoznavanje objektov iz učnega niza Ko je učenje končano, morajo biti vsi objekti iz učnega niza razpoznavni Zgornji pogoj je praviloma dosežen, ko je vsota napak v eni epohi ε epoh pod določeno mejo. e epoh p s 1 n m j 1 i 1 ( ) x si w ji 44
45 Vrhnja plast (VP) z oznakami: W =W win j x j y -prazni nevroni v VP -klastri v VP -konflikti v VP j = y j =5 x n x NET A A A A A A A A A A A K K E K K E KK E K E EE K E E E E E E E E 11x11 n=11 A = Acid E = Ester n y K = Ketone 45
46 Modifikacija Kohonenove UNM Dodatna plast izhodnih nevronov spremeni Kohonenovo v protitočno ( counterpropagation ) umetno nevronsko mrežo Objekt je sestavljen iz opisnega vektorja X (neodvisni) in lastnosti - Y (vektor odvisnih spremenljivk ali tarče) Vhodni nevroni W (vhodna ali Kohonenova plast) Izhodni nevroni O (izhodna ali Grosbergova plast) Določanje zmagovalnega nevrona samo v vhodni plasti glede na podobnost med X in W Popravljanje uteži v obeh plasteh na enak način, glede na razliko med X-W in Y-O 46
47 Kohonenova - vhodna plast w new ji w ( i 1, m j 1, n s 1, p old ji x si w old ji ) Učenje z X o new ji Izhodna plast o ( y i 1, t j 1, n s 1, p old ji si o old ji ) O W j x j y Učenje z Y 47
48 Protitočna (counterpropagation) mreža (CP-UNM) X: reprezentacija molekulske strukture CP ANN j x j y neurons (5 5) i=1 Xi i=644 Y: tarča lastnost P: Napoved lastnost 48
49 Protitočna (counterpropagation) mreža (CP-UNM) X: reprezentacija molekulske strukture CP ANN j x j y neurons (5 5) D j x j y ( ) x w i i i, j x j y i=1 Mapiranje v Kohonenovi plasti Xi Uteži: Wj x j y i Zmagovalni nevron i=644 Y: tarča lastnost P: Napoved lastnost Napoved v Izhodni plasti 49
50 Protitočna (counterpropagation) mreža (CP-UNM) za napoved vezavne afinitete za ERa Y Xj Izhodna plast - ravnina odgovorov HOMO-LUMO energija 50
Supervised (BPL) verses Hybrid (RBF) Learning. By: Shahed Shahir
Supervised (BPL) verses Hybrid (RBF) Learning By: Shahed Shahir 1 Outline I. Introduction II. Supervised Learning III. Hybrid Learning IV. BPL Verses RBF V. Supervised verses Hybrid learning VI. Conclusion
More informationCOMP9444 Neural Networks and Deep Learning 2. Perceptrons. COMP9444 c Alan Blair, 2017
COMP9444 Neural Networks and Deep Learning 2. Perceptrons COMP9444 17s2 Perceptrons 1 Outline Neurons Biological and Artificial Perceptron Learning Linear Separability Multi-Layer Networks COMP9444 17s2
More informationINTELLIGENTNI SISTEMI NEVRONSKE MREŽE IN KLASIFIKACIJA. Nevronske mreže Prof. Jurij F. Tasič Emil Plesnik
INTELLIGENTNI SISTEMI NEVRONSKE MREŽE IN KLASIFIKACIJA Nevronske mreže Prof. Jurij F. Tasič Emil Plesnik 1 Uvod Umetne nevronske mreže ang. Artificial Neural Networks (ANN) Preračunavanje povezav Vzporedno
More informationCounter-propagation neural networks in Matlab
Available online at www.sciencedirect.com Chemometrics and Intelligent Laboratory Systems 90 (2008) 84 91 www.elsevier.com/locate/chemolab Software description Counter-propagation neural networks in Matlab
More informationArtificial Neural Networks The Introduction
Artificial Neural Networks The Introduction 01001110 01100101 01110101 01110010 01101111 01101110 01101111 01110110 01100001 00100000 01110011 01101011 01110101 01110000 01101001 01101110 01100001 00100000
More informationIntroduction Biologically Motivated Crude Model Backpropagation
Introduction Biologically Motivated Crude Model Backpropagation 1 McCulloch-Pitts Neurons In 1943 Warren S. McCulloch, a neuroscientist, and Walter Pitts, a logician, published A logical calculus of the
More informationNeural Networks. Chapter 18, Section 7. TB Artificial Intelligence. Slides from AIMA 1/ 21
Neural Networks Chapter 8, Section 7 TB Artificial Intelligence Slides from AIMA http://aima.cs.berkeley.edu / 2 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural
More informationLast update: October 26, Neural networks. CMSC 421: Section Dana Nau
Last update: October 26, 207 Neural networks CMSC 42: Section 8.7 Dana Nau Outline Applications of neural networks Brains Neural network units Perceptrons Multilayer perceptrons 2 Example Applications
More informationIntroduction to Neural Networks
Introduction to Neural Networks What are (Artificial) Neural Networks? Models of the brain and nervous system Highly parallel Process information much more like the brain than a serial computer Learning
More informationIntroduction to Artificial Neural Networks
Facultés Universitaires Notre-Dame de la Paix 27 March 2007 Outline 1 Introduction 2 Fundamentals Biological neuron Artificial neuron Artificial Neural Network Outline 3 Single-layer ANN Perceptron Adaline
More informationPart 8: Neural Networks
METU Informatics Institute Min720 Pattern Classification ith Bio-Medical Applications Part 8: Neural Netors - INTRODUCTION: BIOLOGICAL VS. ARTIFICIAL Biological Neural Netors A Neuron: - A nerve cell as
More informationNeural networks. Chapter 20, Section 5 1
Neural networks Chapter 20, Section 5 Chapter 20, Section 5 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural networks Chapter 20, Section 5 2 Brains 0 neurons of
More informationCS 4700: Foundations of Artificial Intelligence
CS 4700: Foundations of Artificial Intelligence Prof. Bart Selman selman@cs.cornell.edu Machine Learning: Neural Networks R&N 18.7 Intro & perceptron learning 1 2 Neuron: How the brain works # neurons
More informationNeural Networks. Fundamentals of Neural Networks : Architectures, Algorithms and Applications. L, Fausett, 1994
Neural Networks Neural Networks Fundamentals of Neural Networks : Architectures, Algorithms and Applications. L, Fausett, 1994 An Introduction to Neural Networks (nd Ed). Morton, IM, 1995 Neural Networks
More informationOn the convergence speed of artificial neural networks in the solving of linear systems
Available online at http://ijimsrbiauacir/ Int J Industrial Mathematics (ISSN 8-56) Vol 7, No, 5 Article ID IJIM-479, 9 pages Research Article On the convergence speed of artificial neural networks in
More informationFeedforward Neural Nets and Backpropagation
Feedforward Neural Nets and Backpropagation Julie Nutini University of British Columbia MLRG September 28 th, 2016 1 / 23 Supervised Learning Roadmap Supervised Learning: Assume that we are given the features
More informationAre Rosenblatt multilayer perceptrons more powerfull than sigmoidal multilayer perceptrons? From a counter example to a general result
Are Rosenblatt multilayer perceptrons more powerfull than sigmoidal multilayer perceptrons? From a counter example to a general result J. Barahona da Fonseca Department of Electrical Engineering, Faculty
More informationDEEP LEARNING AND NEURAL NETWORKS: BACKGROUND AND HISTORY
DEEP LEARNING AND NEURAL NETWORKS: BACKGROUND AND HISTORY 1 On-line Resources http://neuralnetworksanddeeplearning.com/index.html Online book by Michael Nielsen http://matlabtricks.com/post-5/3x3-convolution-kernelswith-online-demo
More informationNeural Turing Machine. Author: Alex Graves, Greg Wayne, Ivo Danihelka Presented By: Tinghui Wang (Steve)
Neural Turing Machine Author: Alex Graves, Greg Wayne, Ivo Danihelka Presented By: Tinghui Wang (Steve) Introduction Neural Turning Machine: Couple a Neural Network with external memory resources The combined
More informationNeural networks. Chapter 20. Chapter 20 1
Neural networks Chapter 20 Chapter 20 1 Outline Brains Neural networks Perceptrons Multilayer networks Applications of neural networks Chapter 20 2 Brains 10 11 neurons of > 20 types, 10 14 synapses, 1ms
More informationDepartment of Pharmacy, Annamalai University, Annamalainagar, Tamil Nadu , India, Received
138 Acta Chim. Slov. 2005, 52, 138 144 Scientific Paper Principal Component Artificial Neural Network Calibration Models for Simultaneous Spectrophotometric Estimation of Phenobarbitone and Phenytoin Sodium
More informationMaster Recherche IAC TC2: Apprentissage Statistique & Optimisation
Master Recherche IAC TC2: Apprentissage Statistique & Optimisation Alexandre Allauzen Anne Auger Michèle Sebag LIMSI LRI Oct. 4th, 2012 This course Bio-inspired algorithms Classical Neural Nets History
More informationNONLINEAR CLASSIFICATION AND REGRESSION. J. Elder CSE 4404/5327 Introduction to Machine Learning and Pattern Recognition
NONLINEAR CLASSIFICATION AND REGRESSION Nonlinear Classification and Regression: Outline 2 Multi-Layer Perceptrons The Back-Propagation Learning Algorithm Generalized Linear Models Radial Basis Function
More informationArtificial Neural Networks. Introduction to Computational Neuroscience Tambet Matiisen
Artificial Neural Networks Introduction to Computational Neuroscience Tambet Matiisen 2.04.2018 Artificial neural network NB! Inspired by biology, not based on biology! Applications Automatic speech recognition
More informationArtificial Neural Networks (ANN) Xiaogang Su, Ph.D. Department of Mathematical Science University of Texas at El Paso
Artificial Neural Networks (ANN) Xiaogang Su, Ph.D. Department of Mathematical Science University of Texas at El Paso xsu@utep.edu Fall, 2018 Outline Introduction A Brief History ANN Architecture Terminology
More informationNeural networks. Chapter 19, Sections 1 5 1
Neural networks Chapter 19, Sections 1 5 Chapter 19, Sections 1 5 1 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural networks Chapter 19, Sections 1 5 2 Brains 10
More informationLIMITATIONS OF RECEPTRON. XOR Problem The failure of the perceptron to successfully simple problem such as XOR (Minsky and Papert).
LIMITATIONS OF RECEPTRON XOR Problem The failure of the ercetron to successfully simle roblem such as XOR (Minsky and Paert). x y z x y z 0 0 0 0 0 0 Fig. 4. The exclusive-or logic symbol and function
More informationArtificial Neural Networks. Edward Gatt
Artificial Neural Networks Edward Gatt What are Neural Networks? Models of the brain and nervous system Highly parallel Process information much more like the brain than a serial computer Learning Very
More informationMcGill University > Schulich School of Music > MUMT 611 > Presentation III. Neural Networks. artificial. jason a. hockman
jason a. hockman Overvie hat is a neural netork? basics and architecture learning applications in music History 1940s: William McCulloch defines neuron 1960s: Perceptron 1970s: limitations presented (Minsky)
More informationBack-Propagation Algorithm. Perceptron Gradient Descent Multilayered neural network Back-Propagation More on Back-Propagation Examples
Back-Propagation Algorithm Perceptron Gradient Descent Multilayered neural network Back-Propagation More on Back-Propagation Examples 1 Inner-product net =< w, x >= w x cos(θ) net = n i=1 w i x i A measure
More informationCS 4700: Foundations of Artificial Intelligence
CS 4700: Foundations of Artificial Intelligence Prof. Bart Selman selman@cs.cornell.edu Machine Learning: Neural Networks R&N 18.7 Intro & perceptron learning 1 2 Neuron: How the brain works # neurons
More informationFrequency Selective Surface Design Based on Iterative Inversion of Neural Networks
J.N. Hwang, J.J. Choi, S. Oh, R.J. Marks II, "Query learning based on boundary search and gradient computation of trained multilayer perceptrons", Proceedings of the International Joint Conference on Neural
More informationA Neural Network Scheme For Earthquake. City University, Centre for Information Engineering(1), Department of Computer Science(2),
A Neural Network Scheme For Earthquake Prediction Based On The Seismic Electric Signals Spiros Lakkos 1, Andreas Hadjiprocopis 2, Richard Comley 1, Peter Smith 2 City University, Centre for Information
More informationA Logarithmic Neural Network Architecture for Unbounded Non-Linear Function Approximation
1 Introduction A Logarithmic Neural Network Architecture for Unbounded Non-Linear Function Approximation J Wesley Hines Nuclear Engineering Department The University of Tennessee Knoxville, Tennessee,
More informationLatched recurrent neural network
Elektrotehniški vestnik 7(-2: 46 5, 23 Electrotechnical Review, Ljubljana, Slovenija Latched recurrent neural network Branko Šter University of Ljubljana, Faculty of Computer and Information Science, Laboratory
More informationARTIFICIAL INTELLIGENCE. Artificial Neural Networks
INFOB2KI 2017-2018 Utrecht University The Netherlands ARTIFICIAL INTELLIGENCE Artificial Neural Networks Lecturer: Silja Renooij These slides are part of the INFOB2KI Course Notes available from www.cs.uu.nl/docs/vakken/b2ki/schema.html
More informationIntroduction To Artificial Neural Networks
Introduction To Artificial Neural Networks Machine Learning Supervised circle square circle square Unsupervised group these into two categories Supervised Machine Learning Supervised Machine Learning Supervised
More informationArtificial Neural Networks. Historical description
Artificial Neural Networks Historical description Victor G. Lopez 1 / 23 Artificial Neural Networks (ANN) An artificial neural network is a computational model that attempts to emulate the functions of
More informationNeural Networks (Part 1) Goals for the lecture
Neural Networks (Part ) Mark Craven and David Page Computer Sciences 760 Spring 208 www.biostat.wisc.edu/~craven/cs760/ Some of the slides in these lectures have been adapted/borrowed from materials developed
More informationEEE 241: Linear Systems
EEE 4: Linear Systems Summary # 3: Introduction to artificial neural networks DISTRIBUTED REPRESENTATION An ANN consists of simple processing units communicating with each other. The basic elements of
More informationLearning and Memory in Neural Networks
Learning and Memory in Neural Networks Guy Billings, Neuroinformatics Doctoral Training Centre, The School of Informatics, The University of Edinburgh, UK. Neural networks consist of computational units
More informationArtificial Neural Networks
Artificial Neural Networks 鮑興國 Ph.D. National Taiwan University of Science and Technology Outline Perceptrons Gradient descent Multi-layer networks Backpropagation Hidden layer representations Examples
More informationMachine Learning. Neural Networks. (slides from Domingos, Pardo, others)
Machine Learning Neural Networks (slides from Domingos, Pardo, others) For this week, Reading Chapter 4: Neural Networks (Mitchell, 1997) See Canvas For subsequent weeks: Scaling Learning Algorithms toward
More informationMachine Learning. Neural Networks. (slides from Domingos, Pardo, others)
Machine Learning Neural Networks (slides from Domingos, Pardo, others) Human Brain Neurons Input-Output Transformation Input Spikes Output Spike Spike (= a brief pulse) (Excitatory Post-Synaptic Potential)
More informationChapter ML:VI (continued)
Chapter ML:VI (continued) VI Neural Networks Perceptron Learning Gradient Descent Multilayer Perceptron Radial asis Functions ML:VI-64 Neural Networks STEIN 2005-2018 Definition 1 (Linear Separability)
More informationNetworks of McCulloch-Pitts Neurons
s Lecture 4 Netorks of McCulloch-Pitts Neurons The McCulloch and Pitts (M_P) Neuron x x sgn x n Netorks of M-P Neurons One neuron can t do much on its on, but a net of these neurons x i x i i sgn i ij
More informationNeural Nets in PR. Pattern Recognition XII. Michal Haindl. Outline. Neural Nets in PR 2
Neural Nets in PR NM P F Outline Motivation: Pattern Recognition XII human brain study complex cognitive tasks Michal Haindl Faculty of Information Technology, KTI Czech Technical University in Prague
More informationSimple Neural Nets For Pattern Classification
CHAPTER 2 Simple Neural Nets For Pattern Classification Neural Networks General Discussion One of the simplest tasks that neural nets can be trained to perform is pattern classification. In pattern classification
More informationIntroduction to Neural Networks: Structure and Training
Introduction to Neural Networks: Structure and Training Professor Q.J. Zhang Department of Electronics Carleton University, Ottawa, Canada www.doe.carleton.ca/~qjz, qjz@doe.carleton.ca A Quick Illustration
More informationECE 471/571 - Lecture 17. Types of NN. History. Back Propagation. Recurrent (feedback during operation) Feedforward
ECE 47/57 - Lecture 7 Back Propagation Types of NN Recurrent (feedback during operation) n Hopfield n Kohonen n Associative memory Feedforward n No feedback during operation or testing (only during determination
More informationCourse Overview. before 1943: Neurobiology. before 1943: Neurobiology. before 1943: Neurobiology. The Start of Artificial Neural Nets
Course Overview Introduction to Artificial Neural Networks Kristof Van Laerhoven kristof @ mis.tu-darmstadt.de http://www.mis.informatik.tu-darmstadt.de/kristof We will: Learn to interpret common ANN formulas
More informationMachine Learning. Neural Networks. (slides from Domingos, Pardo, others)
Machine Learning Neural Networks (slides from Domingos, Pardo, others) For this week, Reading Chapter 4: Neural Networks (Mitchell, 1997) See Canvas For subsequent weeks: Scaling Learning Algorithms toward
More informationLecture 4: Feed Forward Neural Networks
Lecture 4: Feed Forward Neural Networks Dr. Roman V Belavkin Middlesex University BIS4435 Biological neurons and the brain A Model of A Single Neuron Neurons as data-driven models Neural Networks Training
More informationARTIFICIAL NEURAL NETWORK PART I HANIEH BORHANAZAD
ARTIFICIAL NEURAL NETWORK PART I HANIEH BORHANAZAD WHAT IS A NEURAL NETWORK? The simplest definition of a neural network, more properly referred to as an 'artificial' neural network (ANN), is provided
More informationCOMPARING PERFORMANCE OF NEURAL NETWORKS RECOGNIZING MACHINE GENERATED CHARACTERS
Proceedings of the First Southern Symposium on Computing The University of Southern Mississippi, December 4-5, 1998 COMPARING PERFORMANCE OF NEURAL NETWORKS RECOGNIZING MACHINE GENERATED CHARACTERS SEAN
More informationARTIFICIAL NEURAL NETWORKS گروه مطالعاتي 17 بهار 92
ARTIFICIAL NEURAL NETWORKS گروه مطالعاتي 17 بهار 92 BIOLOGICAL INSPIRATIONS Some numbers The human brain contains about 10 billion nerve cells (neurons) Each neuron is connected to the others through 10000
More informationRESPONSE PREDICTION OF STRUCTURAL SYSTEM SUBJECT TO EARTHQUAKE MOTIONS USING ARTIFICIAL NEURAL NETWORK
ASIAN JOURNAL OF CIVIL ENGINEERING (BUILDING AND HOUSING) VOL. 7, NO. 3 (006) PAGES 301-308 RESPONSE PREDICTION OF STRUCTURAL SYSTEM SUBJECT TO EARTHQUAKE MOTIONS USING ARTIFICIAL NEURAL NETWORK S. Chakraverty
More informationFundamentals of Neural Networks
Fundamentals of Neural Networks : Soft Computing Course Lecture 7 14, notes, slides www.myreaders.info/, RC Chakraborty, e-mail rcchak@gmail.com, Aug. 10, 2010 http://www.myreaders.info/html/soft_computing.html
More informationArtificial Neural Networks. Part 2
Artificial Neural Netorks Part Artificial Neuron Model Folloing simplified model of real neurons is also knon as a Threshold Logic Unit x McCullouch-Pitts neuron (943) x x n n Body of neuron f out Biological
More informationLab 5: 16 th April Exercises on Neural Networks
Lab 5: 16 th April 01 Exercises on Neural Networks 1. What are the values of weights w 0, w 1, and w for the perceptron whose decision surface is illustrated in the figure? Assume the surface crosses the
More informationFeedforward Networks
Feedforward Networks Gradient Descent Learning and Backpropagation Christian Jacob CPSC 433 Christian Jacob Dept.of Coputer Science,University of Calgary CPSC 433 - Feedforward Networks 2 Adaptive "Prograing"
More informationKeywords- Source coding, Huffman encoding, Artificial neural network, Multilayer perceptron, Backpropagation algorithm
Volume 4, Issue 5, May 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Huffman Encoding
More informationArtificial Neural Networks" and Nonparametric Methods" CMPSCI 383 Nov 17, 2011!
Artificial Neural Networks" and Nonparametric Methods" CMPSCI 383 Nov 17, 2011! 1 Todayʼs lecture" How the brain works (!)! Artificial neural networks! Perceptrons! Multilayer feed-forward networks! Error
More informationGenetic Algorithm Optimized Neural Networks Ensemble. for Estimation of Mefenamic Acid and Paracetamol in Tablets
440 Acta Chim. Slov. 2005, 52, 440 449 Scientific Paper for Estimation of Mefenamic Acid and Paracetamol in Tablets Satyanarayana Dondeti,* Kamarajan Kannan, and Rajappan Manavalan Department of Pharmacy,
More informationFeedforward Networks
Feedforward Neural Networks - Backpropagation Feedforward Networks Gradient Descent Learning and Backpropagation CPSC 533 Fall 2003 Christian Jacob Dept.of Coputer Science,University of Calgary Feedforward
More informationMultilayer Perceptron Tutorial
Multilayer Perceptron Tutorial Leonardo Noriega School of Computing Staffordshire University Beaconside Staffordshire ST18 0DG email: l.a.noriega@staffs.ac.uk November 17, 2005 1 Introduction to Neural
More informationMatrix representation of a Neural Network
Downloaded from orbit.dtu.dk on: May 01, 2018 Matrix representation of a Neural Network Christensen, Bjørn Klint Publication date: 2003 Document Version Publisher's PDF, also known as Version of record
More informationSELECTION OF THE MOST SUCCESFUL NEURAL NETWORK ALGORITHM FOR THE PORPUSE OF SUBSURFACE VELOCITY MODELING, EXAMPLE FROM SAVA DEPRESSION, CROATIA
SELECTION OF THE MOST SUCCESFUL NEURAL NETWORK ALGORITHM FOR THE PORPUSE OF SUBSURFACE VELOCITY MODELING, EXAMPLE FROM SAVA DEPRESSION, CROATIA Marko CVETKOVIĆ 1, Josipa VELIĆ 1 and Filip VUKIČEVIĆ 2 1
More informationFeedforward Networks. Gradient Descent Learning and Backpropagation. Christian Jacob. CPSC 533 Winter 2004
Feedforward Networks Gradient Descent Learning and Backpropagation Christian Jacob CPSC 533 Winter 2004 Christian Jacob Dept.of Coputer Science,University of Calgary 2 05-2-Backprop-print.nb Adaptive "Prograing"
More informationCSC Neural Networks. Perceptron Learning Rule
CSC 302 1.5 Neural Networks Perceptron Learning Rule 1 Objectives Determining the weight matrix and bias for perceptron networks with many inputs. Explaining what a learning rule is. Developing the perceptron
More informationSIMULATION OF FREEZING AND FROZEN SOIL BEHAVIOURS USING A RADIAL BASIS FUNCTION NEURAL NETWORK
SIMULATION OF FREEZING AND FROZEN SOIL BEHAVIOURS USING A RADIAL BASIS FUNCTION NEURAL NETWORK Z.X. Zhang 1, R.L. Kushwaha 2 Department of Agricultural and Bioresource Engineering University of Saskatchewan,
More informationMulti-Dimensional Neural Networks: Unified Theory
Multi-Dimensional Neural Networks: Unified Theory Garimella Ramamurthy Associate Professor IIIT-Hyderebad India Slide 1 Important Publication Book based on my MASTERPIECE Title: Multi-Dimensional Neural
More informationNeural Networks: Introduction
Neural Networks: Introduction Machine Learning Fall 2017 Based on slides and material from Geoffrey Hinton, Richard Socher, Dan Roth, Yoav Goldberg, Shai Shalev-Shwartz and Shai Ben-David, and others 1
More informationArtificial Neural Networks. Q550: Models in Cognitive Science Lecture 5
Artificial Neural Networks Q550: Models in Cognitive Science Lecture 5 "Intelligence is 10 million rules." --Doug Lenat The human brain has about 100 billion neurons. With an estimated average of one thousand
More informationCHALMERS, GÖTEBORGS UNIVERSITET. EXAM for ARTIFICIAL NEURAL NETWORKS. COURSE CODES: FFR 135, FIM 720 GU, PhD
CHALMERS, GÖTEBORGS UNIVERSITET EXAM for ARTIFICIAL NEURAL NETWORKS COURSE CODES: FFR 135, FIM 72 GU, PhD Time: Place: Teachers: Allowed material: Not allowed: October 23, 217, at 8 3 12 3 Lindholmen-salar
More informationCMSC 421: Neural Computation. Applications of Neural Networks
CMSC 42: Neural Computation definition synonyms neural networks artificial neural networks neural modeling connectionist models parallel distributed processing AI perspective Applications of Neural Networks
More informationIn the Name of God. Lecture 9: ANN Architectures
In the Name of God Lecture 9: ANN Architectures Biological Neuron Organization of Levels in Brains Central Nervous sys Interregional circuits Local circuits Neurons Dendrite tree map into cerebral cortex,
More informationArtificial Neural Networks
Artificial Neural Networks CPSC 533 Winter 2 Christian Jacob Neural Networks in the Context of AI Systems Neural Networks as Mediators between Symbolic AI and Statistical Methods 2 5.-NeuralNets-2.nb Neural
More informationNeural Networks Lecturer: J. Matas Authors: J. Matas, B. Flach, O. Drbohlav
Neural Networks 30.11.2015 Lecturer: J. Matas Authors: J. Matas, B. Flach, O. Drbohlav 1 Talk Outline Perceptron Combining neurons to a network Neural network, processing input to an output Learning Cost
More informationMaster Recherche IAC Apprentissage Statistique, Optimisation & Applications
Master Recherche IAC Apprentissage Statistique, Optimisation & Applications Anne Auger Balazs Kégl Michèle Sebag TAO Nov. 28th, 2012 Contents WHO Anne Auger, optimization Balazs Kégl, machine learning
More informationNeural Network to Control Output of Hidden Node According to Input Patterns
American Journal of Intelligent Systems 24, 4(5): 96-23 DOI:.5923/j.ajis.2445.2 Neural Network to Control Output of Hidden Node According to Input Patterns Takafumi Sasakawa, Jun Sawamoto 2,*, Hidekazu
More informationArtificial Neural Networks Examination, March 2004
Artificial Neural Networks Examination, March 2004 Instructions There are SIXTY questions (worth up to 60 marks). The exam mark (maximum 60) will be added to the mark obtained in the laborations (maximum
More informationIntelligent Systems Discriminative Learning, Neural Networks
Intelligent Systems Discriminative Learning, Neural Networks Carsten Rother, Dmitrij Schlesinger WS2014/2015, Outline 1. Discriminative learning 2. Neurons and linear classifiers: 1) Perceptron-Algorithm
More informationConvergence of Hybrid Algorithm with Adaptive Learning Parameter for Multilayer Neural Network
Convergence of Hybrid Algorithm with Adaptive Learning Parameter for Multilayer Neural Network Fadwa DAMAK, Mounir BEN NASR, Mohamed CHTOUROU Department of Electrical Engineering ENIS Sfax, Tunisia {fadwa_damak,
More informationDenoising Autoencoders
Denoising Autoencoders Oliver Worm, Daniel Leinfelder 20.11.2013 Oliver Worm, Daniel Leinfelder Denoising Autoencoders 20.11.2013 1 / 11 Introduction Poor initialisation can lead to local minima 1986 -
More informationChapter ML:VI. VI. Neural Networks. Perceptron Learning Gradient Descent Multilayer Perceptron Radial Basis Functions
Chapter ML:VI VI. Neural Networks Perceptron Learning Gradient Descent Multilayer Perceptron Radial asis Functions ML:VI-1 Neural Networks STEIN 2005-2018 The iological Model Simplified model of a neuron:
More informationArtificial Neural Network and Fuzzy Logic
Artificial Neural Network and Fuzzy Logic 1 Syllabus 2 Syllabus 3 Books 1. Artificial Neural Networks by B. Yagnanarayan, PHI - (Cover Topologies part of unit 1 and All part of Unit 2) 2. Neural Networks
More informationIntroduction and Perceptron Learning
Artificial Neural Networks Introduction and Perceptron Learning CPSC 565 Winter 2003 Christian Jacob Department of Computer Science University of Calgary Canada CPSC 565 - Winter 2003 - Emergent Computing
More informationComputational Intelligence
Plan for Today Single-Layer Perceptron Computational Intelligence Winter Term 00/ Prof. Dr. Günter Rudolph Lehrstuhl für Algorithm Engineering (LS ) Fakultät für Informatik TU Dortmund Accelerated Learning
More informationCan we synthesize learning? Sérgio Hortas Rodrigues IST, Aprendizagem Simbólica e Sub-Simbólica, Jun 2009
Hierarchical Neural Netw rks Can we synthesize learning? Sérgio Hortas Rodrigues IST, Aprendizagem Simbólica e Sub-Simbólica, Jun 2009 Topics Brain review Artificial Neurons Basic Neural Networks Back
More informationλ-universe: Introduction and Preliminary Study
λ-universe: Introduction and Preliminary Study ABDOLREZA JOGHATAIE CE College Sharif University of Technology Azadi Avenue, Tehran IRAN Abstract: - Interactions between the members of an imaginary universe,
More informationWhat Do Neural Networks Do? MLP Lecture 3 Multi-layer networks 1
What Do Neural Networks Do? MLP Lecture 3 Multi-layer networks 1 Multi-layer networks Steve Renals Machine Learning Practical MLP Lecture 3 7 October 2015 MLP Lecture 3 Multi-layer networks 2 What Do Single
More informationCourse 395: Machine Learning - Lectures
Course 395: Machine Learning - Lectures Lecture 1-2: Concept Learning (M. Pantic) Lecture 3-4: Decision Trees & CBC Intro (M. Pantic & S. Petridis) Lecture 5-6: Evaluating Hypotheses (S. Petridis) Lecture
More informationInf2b Learning and Data
Inf2b Learnin and Data Lecture 2 (Chapter 2): Multi-laer neural netorks (Credit: Hiroshi Shimodaira Iain Murra and Steve Renals) Centre for Speech Technolo Research (CSTR) School of Informatics Universit
More informationNeural Networks for Machine Learning. Lecture 2a An overview of the main types of neural network architecture
Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network architecture Geoffrey Hinton with Nitish Srivastava Kevin Swersky Feed-forward neural networks These are
More informationRAINFALL RUNOFF MODELING USING SUPPORT VECTOR REGRESSION AND ARTIFICIAL NEURAL NETWORKS
CEST2011 Rhodes, Greece Ref no: XXX RAINFALL RUNOFF MODELING USING SUPPORT VECTOR REGRESSION AND ARTIFICIAL NEURAL NETWORKS D. BOTSIS1 1, P. LATINOPOULOS 2 and K. DIAMANTARAS 3 1&2 Department of Civil
More informationArtificial Neural Networks
Artificial Neural Networks Short introduction Bojana Dalbelo Bašić, Marko Čupić, Jan Šnajder Faculty of Electrical Engineering and Computing University of Zagreb Zagreb, June 6, 2018 Dalbelo Bašić, Čupić,
More informationReification of Boolean Logic
526 U1180 neural networks 1 Chapter 1 Reification of Boolean Logic The modern era of neural networks began with the pioneer work of McCulloch and Pitts (1943). McCulloch was a psychiatrist and neuroanatomist;
More informationStrength-based and time-based supervised networks: Theory and comparisons. Denis Cousineau Vision, Integration, Cognition lab. Université de Montréal
Strength-based and time-based supervised networks: Theory and comparisons Denis Cousineau Vision, Integration, Cognition lab. Université de Montréal available at: http://mapageweb.umontreal.ca/cousined
More informationARTIFICIAL NEURAL NETWORK APPLICATION OF MODELLING FAILURE RATE FOR BOEING 737 TIRES
ARTIFICIAL NEURAL NETWORK APPLICATION OF MODELLING FAILURE RATE FOR BOEING 737 TIRES Ahmed Z. Al-Garn i and Ahmad Jamal Aerospace Engineering Department King Fahd University of Petroleum and Minerals,
More information