λ-universe: Introduction and Preliminary Study

Transcription

1 λ-universe: Introduction and Preliminary Study ABDOLREZA JOGHATAIE CE College Sharif University of Technology Azadi Avenue, Tehran IRAN Abstract: - Interactions between the members of an imaginary universe, where all the members are adaptive and have learning capability, is simulated numerically by using artificial neural networks. The universe is called λ-universe for brevity. It is shown here that in such a universe, rules governing the behavior of the members might be formed inside the universe by its own members and the randomness which is observed in the behavior of its members is a direct result of the learning capability of the members. Although this is not a simulation of the real universe, some fundamental concepts of astrophysics have been implemented in it. Key-Words: - Neural Networks, Interaction, λ-universe, Adaptivity, Astrophysics, Learning 1 Introduction Learning the cause-effect relationship governing a phenomenon is the main task most researchers expect that artificial learning systems, specifically artificial neural networks, can do well [1,2]. Systems control [3], as one of the most complicated problems in engineering and mechanics, has been well undertaken by neural networks where a neural network is trained to learn to serve as the brain for controlling the system under study [4-7]. In sociology and linguistics too, interactions among intelligent individuals have been simulated recently and it has been shown that states of common knowledge [8] and linguistic rules [9] may emerge as direct results of the exchange of knowledge between the individuals, which is called here the principle of learning from each other. Although learning is mainly considered a high level biological activity in the nature, however, the question raised here is that how would the universe be if every part of it, specifically every part of the matter, had some sort of adaptivity and learning capability? Is it possible that the astrophysical rules that we observe be the outcome of the activities inside the matter itself? One first approach to answer this problem is by numerical simulation. Recently in astrophysics too, computer simulations have provided a better understanding of the stellar evolution and galactic kinematics [10]. Here, results of a preliminary study of the numerical simulation of such a hypothetical universe made up of adaptive members is reported. The members interact with each other and some general rules emerge as a result of their interactions. We may assume that there are some fundamental particles similar to the synaptic transmitters in biological synapses [11,12] which are responsible for this adaptivity and learning of the matter. Some features of this universe, specially the evolution and appearance of rules governing the interactions between particles constituting the matter, are studied here. As an example, the Newton s third law of mechanics, which states the equality of action and reaction forces between two interacting bodies, has been selected for demonstration, where it is shown that this law can be generated by the learning matter itself and also that the 1

2 size of the action and reaction are not a-priori known. Instead, they are determined throughout the time with the aging of the learning universe. This study is not a simulation of the real universe and it was not intended to draw based on this study any conclusion about the evolution of the real universe. However some similarities between this imaginary universe, from its birth and generation to its evolution, and the real universe is noteworthy. It is then conclude that, in such a universe, 1) rules are generated by the matter itself and are ever changing 2) randomness is the direct result of learning capability of the matter, 3) different sets of rules and material properties might evolve in isolated large bodies of such a universe and 4) rules governing and properties of the learning matter depend on its initial conditions of the learning universe. The imaginary learning universe is, for abbreviation purposes and also for its distinguishing from the real universe, denoted by λ-universe, where λ stands for learning. The λ-universe is assumed to be comprised of imaginary fundamental particles, atoms, molecules, and larger bodies such as stellar constellations and galaxies, here denoted by λ- particles, λ-atoms, λ-galaxies, etc. It is assumed here that every subset or member of this universe, called a λ-member, is adaptive whether it is a λ-fundamental particle, a λ-atom or a λ-galaxy. One can easily extend this idea to the interactions between any two members of the λ-universe, for example even between a λ- fundamental particle and a λ-galaxy. 2 Numerical Study To provide some insight about the idea of the λ- Universe, a numerical study was undertaken. The following sections contain information about the assumptions and results of the study. 2.1 Simulation of the λ-members Each adaptive λ-member was simulated by a three layer feed-forward neural network (perceptron) with 1 input, 1 output and 3 hidden units, resulting in 6 connection weights as shown in Fig. 1. This is one of the simplest forms that could be considered for an artificial neural network. The interested reader may refer to any text on the fundamentals of neural networks such as [1] for more details about multi layer feed forward neural networks. Since information in a perceptron is kept in its connection weights, all of the perceptrons which have the same architecture possess the same learning capacity. Hence all the λ-members had the same learning capacity; however they were distinguished from each other by: (1) their activation functions and (2) the values of their connection weights. 2.2 λ-membership Types Then, it was assumed that the activation functions are responsible for the determination of the type of a λ-member. For example, if the member is an atom, its type can be gold, silver, oxygen, etc. In this first attack to the problem, only 3 types of λ-members, denoted by T 1, T 2 and T 3 were assumed existing in our λ- universe. The same activation function was used for all of the processing units in all of the neural networks representing the λ-members of a specific type i as: F i (z)=a i +b i + 2/(1+e -γz ), i=1,2,3 (1) where z is the input to the activation function of a unit, which is equal to the weighted sum of the inputs received by the unit from its preceding units, γ is a parameter which plays a key role in the learning capability of the neural networks [1], assumed a universal constant in this study. Also a i and b i represent two λ-membership type constants. In this study the following arbitrary values of 2

3 γ=0.5, a i =0.1i and b i =0.2i, i=1,2 and 3 (2) were selected. If N = number of λ-members in the λ-universe, and N i = number of λ-members of type T i, i=1,2 and 3, then N= N 1 +N 2 +N 3. The population of λ-members in this first study of the subject was chosen a small one, i.e. N=20. Also it was decided to randomly distribute the λ-members in the λ-membership types T 1, T 2 and T 3, resulting in N 1 =9, N 2 =4 and N 3 = Generation of Initial Connection Weights As Initial Conditions Not all of the individual λ-members, belonging to a specific λ-membership type, were considered exactly the same. They were differentiated by their initial connection weights. At the beginning of the simulation, or in other words right after the birth of our imaginary λ- universe, the values of the connection weights of the λ-members were specified randomly. This was the Big Bang in our imaginary λ- universe, responsible for the generation of variety and randomness in the properties, behavior and rules governing the λ-matter filling the imaginary λ-universe. We have borrowed the terms birth of and Big-Bang from astrophysics, because the numerical process of the generation of the initial connection weights for the λ-members seemed very similar to the early stages of the real universe where the universe was hot, dense, irregular and unisotropic [13,14,15,16]. So, the knowledge content of each λ-member was different form the others in general. However the λ-atoms were bounded to or separated from each other by their λ-membership type numbers. It was included in the simulation that upon the interaction of two λ-members, say λ-members A and B, belonging to the same or two different λ- membership types, each had to receive as input the type number of the other λ-member. The output of each of them was then its response to the other one. Hence in this model of an imaginary universe of λ-matter, a λ-member was known to the other λ-members just by its type number. However it was receiving different reactions from the other λ-members, even from the λ-members belonging to the same type, because of their different knowledge contents. This means that right after the Big Bang in our imaginary universe, each λ-member was following its own rule to respond to the other λ- members. So, rules were local and there was no general rule governing the response of the λ- membership types to one another. 2.4 Interactions Between The λ-members Knowledge Content as One of the properties of the λ-members Similar to the real universe where particles interact through exchanging their properties such as spin, isospin, energy, mass, color, etc. [14], it was assumed in this study that the knowledge content of a λ-member was also one of its properties which had to be exchanged during its interactions with the other λ- members. Hence it was assumed that each two interacting λ-members had to follow the socalled principle of learning from each other which states that any two interacting learning members should try, or are forced, to reduce the difference in their mutual responses through the gradual updating of their connection weights [8]. To this end, each λ-member was using as its target output the output of the other λ-member Learning Rule and Updating of Connection Weights In this simulation, the back-propagation rule of learning [17] was used for the updating of connection weights. Interactions among λ- members occured during discrete time steps where at each time step only two λ-members 3

4 were selected randomly from the universe of the λ-members, their responses to each other were computed, and using a small appropriate learning rate, here lr=0.0002, their connection weights were updated so that their outputs got closer to each other and the difference between their outputs was reduced by a small amount. A schematic representation of this concept of interaction and knowledge exchange between two λ-members is shown in Fig Results of Numerical Simulations Numerical simulations, based on the above mentioned assumptions about the λ-matter and also the methodology of simulating interactions between the λ-members, show that after quite a large number of updating cycles, local rules disappear and more general rules emerge. These general rules are specifying the type to type response of the λ-members. To provide a more quantitative understanding of the evolution process, for each two types i,j = 1,2 and 3, the mean and standard deviation of the rules governing the type to type responses were monitored thorough the time. To this end, at each simulation time step, type number j was fed as input to all of the λ-members of type i and the mean E ij and the standard deviation S ij of the observed outputs were calculated. As an example, the following E and S matrices which have been calculated right after the Big Bang as well as after a large number of interactions, i.e. a total of 10,0000,000 interactions among all of the λ-members, are reported here: E 0 = (3) S 0 = (4) E L = (5) S L = (6) where subscripts 0 and L mean after 0 and after a large number of interactions respectively. It is noteworthy that the matrix E 0 =[ E ij ] 0 has not been symmetric at the onset of interactions. However with the advent of time and more interactions, it has got closer to a symmetric form which is of course an evidence for the gradual appearance of the third Newton s law. Another point to notice here, is the gradual convergence of the S matrix to 0 as the imaginary universe has got older and older. It means that with aging, the rules in this universe have become more and more refined, simpler and universal. As can be seen, all the elements of the S matrix have converged to 0 while E has changed dramatically but has not converged to 0. The convergence of S has been, according to the numerical observations, essentially monotonic as represented in Fig. 3 where each curve shows the variation of one of the 9 elements of the S matrix as a function of the simulation time step for up to 10,000,000 time steps. 3.1 Effect of Initial Conditions Next it was desired to study the effect of the initial conditions of the λ-universe on the final results of the interactions. Further numerical studies revealed that in most of the situations the E and S matrices have converged to symmetry and 0 respectively, although E has been different for different initial conditions in general. However there have also been many initial conditions for which such convergence has not been observed even after a large number of interactions, which may be considered as an indication of the probable instability of the 4

5 generated λ-universe. Hence the initial conditions of the λ-universe plays a vital role in its stability and the development of its rules. More research is being done on more complicated situations where the input vector to a λ-member is many dimensional, containing information for example about the position, mass and other properties attached with the λ- member itself as well as the other λ-members which are interacting with it. Also a many dimensional output can be considered for a λ- member, accounting for the exchange of its different properties with the other λ-members. Finally the architecture of a λ-member can be considered a function of its λ-membership type. 4. Concluding Remarks The results obtained in this numerical study of the imaginary universe of λ-members can now be summarized as: (1) If not all, some of the general rules such as the rule of equality of action and reaction governing the λ-members may emerge as the results of interactions in the λ-universe. These rules which are changing through out the time are generated by the learning matter itself, and are not imposed on it from the beginning. (2) Rules governing a λ-universe are generally functions of the initial conditions of the λ- matter. In this study, the randomly generated connection weights of the neural networks have served as the initial conditions of the λ- matter. This numerical process of generating randomness in the λ-matter can be considered as a Big-Bang in theλ-universe. (3) The randomness which is observed in the response of the members belonging to a specific λ-membership type to the λ- members of the another type is a direct result of the difference in their knowledge content. The S L matrix in equation (6) which is not exactly 0 supports this statement. References [1] D. E. Rumelhart and J.L. Mc Clelland, Parallel Distributed Processing, V.1: Foundations, MIT Press, Cambridge MA, [2] K. Kang, J. H. Oh and C. Kwon, Learning by a Population of Perceptrons, Phys. Rev. E, vol.55, 1997, pp [3] L. Meirovitch, Dynamics and Control of Structures, Wiley, New York, [4] A. Joghataie, Neural Network and Fuzzy Logic in Structural Control, Ph.D. thesis, University of Illinois at Urbana-Champaign, [5] R. Bakker, J. Schouten, F. Takens and C.M. van den Bleek, Neural Network Model to Control an Experimental Chaotic Pendulum, Phys. Rev. E,54, 1996, pp [6] E.R. Weeks and J.M. Burgess, Evolving Artificial Neural Networks to Control Chaotic Systems, Phys. Rev. E, vol. 56, 1997, pp [7] D.A. White and D.A. Sofge, Handbook of Intelligent Control, Van Nostrand Reinhorn, New York, [8] A. Joghataie, Communication in a Society of Interacting Perceptrons, Proceedings of IEEE International Conference on Systems, man and Cybernetics, SMC 99, Tokyo, Japan, October 12-15, 1999, pp. V-149 to V-153. [9] E. Hutchins and B. Hazelhurst, How to invent a lexicon: the development of shared symbols in interaction, in Artificial Societies, the Computer Simulation of Life, Edited by N. Gilbert, UCL Press Ltd., [10] J.M.A. Danby, R. Kouzes and C. Witney, Astrophysics Simulations, Wiley, New York,1995. [11] G. M. Shepherd, Neurobiology, Oxford Univ. Press, New York, [12] J.A. Anderson E. Rosenfeld, Neurocomputing: Foundations of Research, MIT Press, Cambridge, [13] J. Silk, The Big Bang: the Creation and 5

6 Evolution of the Universe, W. H. Freedman and Co., San Francisco, [14] I.L. Rosental, Big Bang Big Bounce: How Particles and Fields Drive Cosmic Evolution, English Translation, Springer Verlag, Berlin Heidelberg, [15] A. Fairall, Large Scale Structures in the Universe, Wiley-Praxis, Chichester, [16] R. Ellis, The Formation and Evolution of Galaxies, Nature, vol. 395, 1998, pp. A3- A8. [17] D.E. Rumelhart, G.E. Hinton and R.J. Williams, Learning Representations by Backpropagating Errors, Nature, vol. 323, 1996, pp Input Output FIG. 1. Architecture of the multi-layer feedforward neural networks used in this study Standard Deviations, Sij, I,j=1,2, Simulation Time Steps j Member: A, type :i i FIG. 3 Each Curve is Related to one of the 9 Elements of the S matrix Member: B, type :j i,j = 1,2,3 FIG.2. Interaction and exchange of information between two λ-members 6

7 7