Complete Recall on Alpha-Beta Heteroassociative Memory

Transcription

1 Complete Recall on Alpha-Beta Heteroassociative Memory Israel Román-Godínez and Cornelio Yáñez-Márquez Centro de Investigación en Computación Juan de Dios Bátiz s/n esq. Miguel Othón de Mendizábal Unidad Profesional Adolfo López Mateos Del. Gustavo A. Madero, México, D.F. México Abstract. Most heteroassociative memories models intend to achieve the recall of the entire trained pattern. The Alpha-Beta associative memories only ensure the correct recall of the trained patterns in autoassociative memories, but not for the heteroassociative memories. In this work we present a new algorithm based on the Alpha-Beta Heteroassociative memories that allows, besides correct recall of some altered patterns, perfect recall of all the trained patterns, without ambiguity. The theoretical support and some experimental results are presented. 1 Introduction Associative memories have been an active area for research in computer sciences. In this respect, computer scientists are interested in developing mathematical models that behave as similar as possible to associative memories and, based on the former models, create, design and operate systems that are able to learn and recall patterns [1-3]. The ultimate goal of an associative memory is to correctly recall complete patterns from input patterns. These patterns might be an altered version of the one used to create the associative memory. The first known mathematical model of an associative memory is Steinbuch s Lernmatrix, developed in 1961 [4]. In the following years, many efforts were made. By 1982, Hopfield created a model that works, simultaneously, as associative memory and a neural network [5]. In the late 1990s morphological associative memories were developed by Ritter et al. [6]. In 2002, a more efficient model of associative memories arose; the Alpha-Beta associative memories were inspired on morphological associative memories [1]. Until this day, the Alpha-Beta model has been applied to several noteworthy problems, such as automatic color matching [7] and lenguage translators [8]. In this paper we propose an improvement on the Alfa-Beta associative memories, particularly on the heteroassociative memory, to ensure the correct recall of the fundamental set, characteristic that does not have the original model. The mathematical support is presented. A. Gelbukh and A.F. Kuri Morales (Eds.): MICAI 2007, LNAI 4827, pp , c Springer-Verlag Berlin Heidelberg 2007

2 194 I. Román-Godínez and C. Yáñez-Márquez This paper is organized as follows. Sections 2 is focused on explaining the Alpha-Beta heteroassociative memory model. Section 3 contains the core proposal and its theoretical support. Section 4 is devoted to the experimental results and finally the Section 5 is about conclusions and future research. 2 Alpha-Beta Associative Memories Here we use basic concepts about associative memories presented in [1]. An associative memory M is a system that relates input patterns, and outputs patterns, as follows: x M y. Each input vector x forms an association with a corresponding output vector y.thek-th association will be denoted as ( x k, y k).associative memory M is represented by a matrix whose ij-th component is m ij, and is generated from an apriori finite set of known associations, called the fundamental set of associations. If μ is an index, the fundamental set is represented as: {(x μ, y μ ) μ = 1, 2,..., p} with p the cardinality of the set. The patterns that form the fundamental set are called fundamental patterns. If it holds that x μ = y μ μ {1, 2,..., p}, thenm is autoassociative, otherwise it is heteroassociative. In this latter case it is possible to establish that μ {1, 2,..., p} for which x μ y μ. If when feeding a unkown fundamental pattern x ω with ω {1, 2,..., p} to an associative memory M, it happens that the output corresponds exactly to the associated pattern y ω, we say that recall is correct. Theheartofthemathematicaltoolsused in the Alpha-Beta model, are two binary operators designed specifically for these memories. These operators are defined in [1] as follows: First, we define the sets A = {0, 1} and B = {0, 1, 2}, then the operators α : A A B and β : A B A are defined in tabular form: Table 1. Alpha and Beta Operators xyα(x, y) x y β(x, y) Two types of heteroassociative Alpha-Beta memories are proposed: type Max ( ) andtypemin( ). For the generation of both types we will use the operator, which has the following [ form, for indices μ {1, 2,...,p},i {1, 2,...,m}, and j {1, 2,...,n}: y μ (x μ ) t] = α ( y μ ) i ij,xμ j. Alpha-Beta Heteroassociative Memories Type Max Learning[ Phase. For every μ = 1, 2,...,p, from the pair (x μ, y μ ) build the matrix: y μ (x μ ) t] Applying the Max binary operator,thev matrix m n

3 Complete Recall on Alpha-Beta Heteroassociative Memory 195 is: V = [ p y μ (x μ ) t] then the ij-th entry is given as: ν ij = p α(yμ i,xμ j ). We can observe that v ij B, i {1, 2,...,m}, j {1, 2,...,n}. Recall Phase. A pattern x ω,that could be or not from the fundamental set, is presented to the heteroassociative Alpha-Beta memory of type and we do the operation Δ β : VΔ β x ω. The result is a column vector of dimension m, whose i-th component is: (VΔ β x ω ) i = n β(ν ij,x ω j ). Remark 1. The Alpha-Beta Heteroassociative Memory Type Min are developed by duality, based on the learning and recall phase of the Alpha-Beta heteroassociative memories type Max. Wherever there is a operator change it for,if there is an change it for, where the operator Δ β is used change it for β. 3 Alpha-Beta Heteroassociative Memories with Complete Recall The Alpha-Beta autoassociative memories guarantee the complete recall of the fundamental set [20], but in the case of the heteroassociative memories is not possible to ensure this behavior. In this section we propose a new algorithm, modifying the original, with which the complete recall of the fundamental set is guaranteed. Definition 1. Let h, n Z +,A = {0, 1} and let x h A n be a binary pattern. We denote the sum of the positive components of x h by: U h = x h j. Definition 2. Let V be an Alpha-Beta heteroassociative memory type Max and {(x μ, y μ ) μ =1, 2,..., p} its fundamental set with x μ A n and y μ A p,a = {0, 1},B = {0, 1, 2},n Z +. The sum of the components with value equal to one of the i-th row of V is given as: s i = T j where T B n and its components { 1 νij =1 are defined as: T i = 0 ν ij 1 j {1, 2,..., n} and the s i components conform the max sum vector with s Z p. Definition 3. Let α, β, n Z +,A= {0, 1} and let x α, x β A n be two vectors; then x α x β x α i =1 x β i =1 i {1, 2,...,n} and x α < x β ix α i x β i and j such that x α j <xβ j Definition 4. Let x α A n with α, n Z +,A = {0, 1} ; each { component of 1 x the negated vector, denoted by x α,ofx α is given as: x α α i = i =0 0 x α i =1 i {1, 2,...,n}. Definition 5. Let α, β, n Z +,A = {0, 1} and let be x h A n. We denote the sum of the components equal to 0 of x h as:c h = x h i. i=1

4 196 I. Román-Godínez and C. Yáñez-Márquez Definition 6. Let Λ be a Alpha-Beta heteroassociative memory type Min and {(x μ, y μ ) μ =1, 2,..., p} its fundamental set with x μ A n and y μ A p,a = {0, 1},B = {0, 1, 2},n Z +. The sum of the components with value equal to zero of the i-th row of Λ is given as:r i = n T j where T B n and its components { 1 λij =0 are defined as: T i = 0 λ ij 0 j {1, 2,..., n} and the r i components conform the min sum vector with r Z p. Alpha-Beta Heteroassociative Memory Type Max Learning Phase. Let x A n and y A p be an input and output vectors, respectively. The corresponding fundamental set is denoted by {(x μ, y μ ) μ = 1, 2,..., p} which is built according with the following conditions: the y vectors are built with the one-hot codification: assigning for y μ the following values: y μ k =1,andyμ j =0forj =1, 2,...,k 1,k+1,...,m where k {1, 2,...,m}. And to each y μ vector correspond one and only one x μ vector. Step 1. For each μ {1, 2,...p}, from the couple (x μ, y μ ) build the matrix: [y μ (x μ ) t ] m n then, the min binary operator is applied to the matrices. p Therefore, the V matrix is obtained as follow: V = [y μ (x μ ) t ]wherethe ij-th component is given by: v ij = Recalling Phase p α(y μ i,xμ j ). Step 1. A pattern x ϖ is presented to V, the Δ β operation is done and the resulting vector is assigned to a vector called z ϖ : z ϖ = VΔ β x ϖ. Then the i-th n component of z ϖ is: z ϖ i = β(v ij,x ϖ j ) Step 2. Once we have the V matrix, it is necesary to build the max sum vector s according to the definition 2, therefore the corresponding y ϖ is given as: 1 if s i = k z yi ϖ i = k θs ϖ =1 0 otherwise where θ = {i z ϖ i =1}. Below are presented the lemmas, and a theorem that support the Alpha-Beta heteroassociative memory type Max presented before. Lemma 1. Let x i A n be a pattern randomly chosen from the fundamental set. In the new Alpha-Beta heteroassociative memory type Max learning phase, x i contributes, only, at the i-th row of V with U i times the value 1 and (n U i ) times the value 0.

5 Complete Recall on Alpha-Beta Heteroassociative Memory 197 Proof. Let x h A n and y h A p with A = {0, 1} and k, n, p Z + be two fundamental patterns, randomly chosen, that form the k-th association (x k,y k ) of V. According with the learnning phase we know that the matrix V is given p by: V = [y μ (x μ ) t ]particularlythek-th association is: [ y k (x k ) t] ij = α(yi k, xk j ). Now, by how the vector yk has been built, it happend that i {1, 2,..., k 1,k+1,...p}, j {1, 2,...n},k {1, 2,...p} yk k =1 α(yk k, xk j )=1 α(yk k, xk j )=2 yi k =0 α(yk i, xk j )=1 α(yk i, xk j )=0 (1) according to expression 1 it is evident that the maximum values of the k-th matrix are stored in its k-th row, depending exclusively on the values of x k,in other words, when x k j =1 α(yk k, xk j )=1orxk j =0 α(yk k, xk j )=2. Therefore, considering that for every fundamental association, to each input pattern correspond one and only one output pattern and that k with k {1, 2,...p} was randomly chosen; we can ensure that V is affected in its i-th row by x i and it is affected with U i times the value 1 and (n U i )timesthevalue2. Thus, the components of the V matrix contain only the values 1 or 2. Finally, we can rewrite the learning phase as follow: i {1, 2,..., p}, j {1, 2,...n} v ij = α(y i i, xi j ) (2) Lemma 2. Let s be the max sum vector of the matrix V; then s i = U i i {1, 2,...p} Proof. Let s be the max sum vector of the matrix V. Itsi-th component is expressed as definition 2 s i = T j (3) In the other hand, we know by definition 1 that U h = x h j.particularly,fora i with i {1, 2,...p} theexpressioncouldbewritenasfollow: U i = x i j (4) Moreover, we know by lemma 1 in expression 2 that x i affects the matrix V only in its i-th row, so it is possible to rewrite the expression 3 as: s i = α(yi, i xi j ). Given that yi i =1 i {1, 2,...p} and that α(yi i, xi j ) depends on xi j, then according to lemma 1 s i = x i j (5)

6 198 I. Román-Godínez and C. Yáñez-Márquez Finally, by transitivity of the equations 4 and 5 we can conclude that s i = x i j = Ui. Lemma 3. Let V be a heteroassociative memory type Max which fundamental set is {(x μ, y μ ) μ =1, 2,...p}, thenletx ϖ A n be pattern that will be presented to V with A {0, 1},ϖ {1, 2,..., p},n,p Z +. The z ϖ A p vector obtained from the original Alpha-Beta heteroassociative recall phase type Max will contain the value 1 in its i-th component where the i-th row of V correspond to the fundamental patterns lower or equal to x ϖ ;putdifferently: izi ϖ =1 x i x ω,x i {(x μ, y μ ) μ =1, 2,...p}. Proof. According with the original Alpha-Beta heteroassociative memory recall phase type Max we know that: in order to z ϖ i =1 n β(v ij,x ϖ j )=1 (6) n β(v ij,x ϖ j ) = 1, due to β only produce 1 or 2 values, it is necesary that j {1, 2,..., n} β(v ij,x ϖ j )=1. Therefore, considering lemma 1, j {1, 2,..., n} just the following cases are possible { β(v ij,x ϖ v ij =1 x ϖ j =1 j )=1 v ij =2 (x ϖ j =1 x ϖ (7) j =0) Now, as lemma 1 says, each x i pattern affect only the i-th row of V and it does acording to learning phase, from the expression 7 we can infer that: x i = x ϖ if j always happend that (v ij =1 x ϖ j =1)or(v ij =2 x ϖ j =0)andx i <x ϖ if j always happend that (v ij =1 x ϖ j =1)or j(v ij =2 x ϖ j =1). This is therefore, by transitivity of 6,7,8 we can conclude: x i x ϖ (8) iz ϖ i =1 x i x ϖ,x i {(x μ, y μ ) μ =1, 2,...p} Theorem 1. Let V be a heteroassociative memory type Max which fundamental set is {(x μ, y μ ) μ =1, 2,...p},without any pair repeted.let x ϖ A n be an input pattern presented to V and z ϖ A p the resulting class vector from the original Alpha-Beta heteroassociative memory recall phase type Max. The proposed algorithm will always obtain complete recall, in other word, we will always obtain the corresponding y ϖ without ambiguity. Proof. To prove the complete recall of the proposed algorithm it would be necesary to ensure that, for all components where zi ϖ = 1 there is just one maximum value in the s i components and it correspond to the correct pattern.this could

7 Complete Recall on Alpha-Beta Heteroassociative Memory 199 be demostrated by contradiction. Let x α A n and x β A n be the corresponding patterns to z ϖ α =1andzϖ β =1whenxϖ is presented to V with x α,x β,x ϖ {(x μ, y μ ) μ =1, 2,...p}.First, we assume that x α is the correct pattern and x β is an arbitrary spurios recalled pattern with corresponding s i values s α and s β, respectively and it holds that s α >s β. Now, we assume the negated of what we want to prove. s α s β (9) Now, by lemma 2, we know that the expression 9 could be writen as follow: U α U β (10) By lemma 3 for each spurious pattern x i, where x ϖ is the correct, imply that izi ϖ =1, x i x ϖ x i < x ϖ. Therefore, we can take as a hypothesis: x β < x α (11) according to definition 1, the inequality 11 could be expressed as U β <U α which is a contradiction with expression 10, then U α U β is false. Therefore, U α >U β, put differently s α >s β, is true for every spurious recalled pattern since x β was chosen arbitrarily. Alpha-Beta Heteroassociative Memory Type Min Learning Phase. Let x A n and y A p be input and output vectors, respectively. The corresponding fundamental set is denoted by {(x μ, y μ ) μ = 1, 2,..., p}. which is built according with the following conditions: the y vectors are built with the zero-hot codification: assigning for the output binary pattern y μ the following values: y μ k =0,andyμ j =1forj =1, 2,...,k 1,k+1,...,m where k {1, 2,...,m}. And, to each y μ vector correspond one and only one x μ vector. Step 1. For each μ {1, 2,...p}, from the couple (x μ, y μ ) build the matrix: [y μ (x μ ) t ] m n then the Min binary operator ( ) is applied to the matrices p obtained. Therefore, the Λ matrix is obtained as follow: Λ = [y μ (x μ ) t ] where the ij-th component is given by: λ ij = Recalling Phase p α(y μ i,xμ j ). Step 1. A pattern x ϖ is presented to Λ, the β operation is done and the resulting vector is assigned to a vector called z ϖ : z ϖ = Λ β x ϖ The i-th component n of the resulting column vector are: z ϖ i = β(λ ij,x ϖ j )

8 200 I. Román-Godínez and C. Yáñez-Márquez Step 2. It is necesary to build the min sum vector r according to the definition 6, therefore the corresponding y ϖ is given as: 0 if r i = k z yi ϖ i = k θr ϖ =0 1 otherwise where θ = {i z ϖ i =0}. Below are presented the lemmas, and a theorem that support the Alpha-Beta heteroassociative memory type Min presented before. Due to a matter of space, the proof of the lemma 4,5 and 6 are not developed here, but they were obtained by duality. Lemma 4. Let x i A n be a pattern randomly chosen from the fundamental set. In the Alpha-Beta heteroassociative memory type Min learning phase, x i contributes, only, at the i-th row of Λ with U i times the value 0 and (n U i ) times the value 1. Proof. This proof is similar to the one presented on lemma 1 taking account the conditions expressed on remark 1. Lemma 5. Let r be the min sum vector of the matrix Λ; then r i = C i, i {1, 2,...p}. Proof. This proof is similar to the one presented on lemma 2 taking account the conditions presented on remark 1. Lemma 6. Let Λ be a heteroassociative memory type Min which fundamental set is {(x μ, y μ ) μ =1, 2,...p}. Letx ϖ A n be an input pattern that will be presented to Λ with A {0, 1},ϖ {1, 2,..., p},n,p Z +. The z ϖ A p vector is obtained from the original Alpha-Beta heteroassociative recall phase type Min. This vector will contain the value 0 in its i-th component where the i-th row of Λ correspond to the fundamental patterns greater or equal to x ϖ ;put differently: izi ϖ =0 x i x ω,x i {(x μ, y μ ) μ =1, 2,...p}. Proof. This proof is similar to the one presented on lemma 3 taking account the conditions expressed on remark 1. Theorem 2. Let Λ be a heteroassociative memory type Min which fundamental set, without any pair repeted, is {(x μ, y μ ) μ =1, 2,...p}.Let x ϖ A n be an input pattern presented to Λ and z ϖ A p the resulting class vector from the original Alpha-Beta heteroassociative memory recall phase type Min. The proposed algorithm will always obtain complete recall, in other word, we will always obtain the corresponding y ϖ without ambiguity. Proof. To prove the complete recall of the proposed algorithm it would be necesary to ensure that, for all components where zi ϖ = 0 there is just one maximum value in the r i components and it correspond to the correct pattern.this could

9 Complete Recall on Alpha-Beta Heteroassociative Memory 201 be demostrated by contradiction. Let x α A n and x β A n be the corresponding patterns to z ϖ α =0andzϖ β =0whenxϖ is presented to Λ with x α,x β,x ϖ {(x μ, y μ ) μ =1, 2,...p}.First, we assume that x α is the correct pattern and x β is an arbitrary spurios recalled pattern with corresponding r i values r α and r β, respectively and it holds that r α <r β. Now, we assume the negated of what we want to prove. s α s β (12) Now, by lemma 5, we know that the expression 12 could be writen as follow: C α C β (13) By lemma 6 for each spurious pattern x i, where x ϖ is the correct, imply that izi ϖ =0, x i x ϖ x i > x ϖ. Therefore, we can consider as a hypothesis: x β > x α (14) according to definition 5, the inequality 14 could be expressed as U β >U α which is a contradiction with expression 13, then C α C β is false. Therefore, C α <C β, put differently r α >r β, is true for every spurious recalled pattern since x β was chosen arbitrarily. 4 Experimental Results In despite of the theoretical support presented in the last section, a serie of experiments were done to illustrate the efficiency of our proposal. With n the dimension of the input vector and p the number of input patterns, three different finite samples were randomly generated, automatically. Each of them was used to build the six different fundamental sets according with the specification mentioned in our proposal. After that, the six different memories -three Max and three Min- were built and the recall phase, each one with its corresponding fundamental patterns, was applied. The results are presented in table 2. It is evident that the original algorithm presents more errors than the one proposed in this paper. Experiment Number Table 2. Experimental Results n p Original Algorithm Error (%) Max Min Max Min Modified Algorithm Error (%)

10 202 I. Román-Godínez and C. Yáñez-Márquez 5 Conclusion and Future Work In this work we proposed a new algorithm for the Alpha-Beta heteroassociative memories that let us recall the fundamental patterns without ambiguity. Therefore, the model of Alpha-Beta associative memories ensure the complete recall for the fundamental set in both cases. However, the conditions for this correct recall on non-fundamental patterns has not yet characterized. The theoretical support for this proposal is presented here along with some experimental test. Currently we are working on applications of the new Alpha-Beta heteroassociative algorithm and as a future work, we will investigate which are the condition that allow our algorithm to show correct recall on non-fundamental patterns. Acknowledgements. The authors would like to thank the Instituto Politécnico Nacional (Secretaría Académica, COFAA, SIP, and CIC), the CONACyT, and SNI for their economical support to develop this work. References 1. Yáñez-Márquez, C.: Associative Memories Based on Order Relations and Binary Operators (in Spanish). PhD Thesis. Center for Computing Research, México (2002) 2. Kohonen, T.: Self-Organization and Associative Memory. Springer, Heidelberg (1989) 3. Hassoun, M.H.: Associative Neural Memories. Oxford University Press, New York (1993) 4. Steinbuch, K.: Die Lermatrix, Kybernetik 1(1), (1961) 5. Hopfield, J.J.: Neural networks and physical systems with emergent collective computa-tional abilities. In: Proceedings of the National Academy of Sciences, vol. 79, pp (1982) 6. Ritter, G.X., Sussner, P., Diaz-de-Leon, J.L.: Morphological Associative Memories. IEEE Transactions on Neural Networks. 9, (1998) 7. Yáñez-Márquez, C., Felipe-Riverón, E.M., López-Yáñez, I., Flores-Carapia, R.: A Novel Approach to Automatic Color Matching, Lecture Notes in Computer Science. In: Martínez-Trinidad, J.F., Carrasco Ochoa, J.A., Kittler, J. (eds.) CIARP LNCS, vol. 4225, pp Springer, Heidelberg (2006) 8. Acevedo-Mosqueda, M.E., Yáñez-Márquez, C., López-Yáñez, I.: Alpha-Beta Bidirectional Associative Memories Based Translator. IJCSNS International Journal of Computer Science and Network Security 6(5A), (2006)