Geometric Analogue of Holographic Reduced Representations

Transcription

1 Geometric Analogue of Holographic Reduced Representations Agnieszka Patyk Ph.D. Supervisor: prof. Marek Czachor Faculty of Applied Physics and Mathematics, Gdańsk University of Technology, Poland and Centrum Leo Apostel (CLEA), Vrije Universiteit Brussel, Belgium Grimma, August 2008

2 Outline Background Distributed representations Previous architectures GA model Binary parametrization of the geometric product Cartan representation Example Signatures Test results Recognition tests Blade linearity Future work Computer program CartanGA

3 Distributed representations Example

4 Distributed representations Example green color + triangular shape = green triangle

5 Distributed representations Example green color + triangular shape = green triangle Idea

6 Distributed representations Example green color + triangular shape = green triangle Idea An opposite and an alternative of traditional data structures (lists, databases, etc...).

7 Distributed representations Example green color + triangular shape = green triangle Idea An opposite and an alternative of traditional data structures (lists, databases, etc...). Hinton (1986):

8 Distributed representations Example green color + triangular shape = green triangle Idea An opposite and an alternative of traditional data structures (lists, databases, etc...). Hinton (1986): each concept is represented over a number of units (bytes)

9 Distributed representations Example green color + triangular shape = green triangle Idea An opposite and an alternative of traditional data structures (lists, databases, etc...). Hinton (1986): each concept is represented over a number of units (bytes) each unit participates in the representation of some number of concepts

10 Distributed representations Example green color + triangular shape = green triangle Idea An opposite and an alternative of traditional data structures (lists, databases, etc...). Hinton (1986): each concept is represented over a number of units (bytes) each unit participates in the representation of some number of concepts the size of a distributed representation is usually fixed

11 Distributed representations Example green color + triangular shape = green triangle Idea An opposite and an alternative of traditional data structures (lists, databases, etc...). Hinton (1986): each concept is represented over a number of units (bytes) each unit participates in the representation of some number of concepts the size of a distributed representation is usually fixed the units have either binary or a continuous-space values

12 Distributed representations Example green color + triangular shape = green triangle Idea An opposite and an alternative of traditional data structures (lists, databases, etc...). Hinton (1986): each concept is represented over a number of units (bytes) each unit participates in the representation of some number of concepts the size of a distributed representation is usually fixed the units have either binary or a continuous-space values data patterns are chosen as random vectors or matrices

13 Distributed representations Example green color + triangular shape = green triangle Idea An opposite and an alternative of traditional data structures (lists, databases, etc...). Hinton (1986): each concept is represented over a number of units (bytes) each unit participates in the representation of some number of concepts the size of a distributed representation is usually fixed the units have either binary or a continuous-space values data patterns are chosen as random vectors or matrices in most distributed representations only the overall pattern of activated units has a meaning (resemblance to a multi-superimposed photograph)

14 Distributed representations Atomic objects and complex statements

15 Distributed representations Atomic objects and complex statements bite agent Fido + bite object Pat = Fido bit Pat }{{}}{{} chunk chunk roles: bite agent, bite object fillers: Fido, Pat

16 Distributed representations Atomic objects and complex statements bite agent Fido + bite object Pat = Fido bit Pat }{{}}{{} chunk chunk roles: bite agent, bite object fillers: Fido, Pat name Pat + sex male + age 66 = PSmith roles: name, sex, age fillers: Pat, male, 66

17 Distributed representations Atomic objects and complex statements bite agent Fido + bite object Pat = Fido bit Pat }{{}}{{} chunk chunk roles: bite agent, bite object fillers: Fido, Pat name Pat + sex male + age 66 = PSmith roles: name, sex, age fillers: Pat, male, 66 see agent John + see object Fido bit Pat = John saw Fido bit Pat Roles are always atomic objects but fillers may be complex statements.

18 Distributed representations Atomic objects and complex statements bite agent Fido + bite object Pat = Fido bit Pat }{{}}{{} chunk chunk roles: bite agent, bite object fillers: Fido, Pat name Pat + sex male + age 66 = PSmith roles: name, sex, age fillers: Pat, male, 66 see agent John + see object Fido bit Pat = John saw Fido bit Pat Roles are always atomic objects but fillers may be complex statements. Binding and chunking binding + superposition (also called chunking)

19 Distributed representations Decoding

20 Distributed representations Decoding decoding x 1 (approximate/pseudo) inverse

21 Distributed representations Decoding decoding x 1 (approximate/pseudo) inverse Examples

22 Distributed representations Decoding decoding x 1 (approximate/pseudo) inverse Examples What is the name of PSmith? (name Pat + sex male + age 66) name 1 = Pat + noise Pat

23 Distributed representations Decoding decoding x 1 (approximate/pseudo) inverse Examples What is the name of PSmith? (name Pat + sex male + age 66) name 1 = Pat + noise Pat What did Fido do? (bite agent Fido + bite object Pat) Fido 1 = bite agent + noise bite agent

24 Distributed representations Decoding decoding x 1 (approximate/pseudo) inverse Examples What is the name of PSmith? (name Pat + sex male + age 66) name 1 = Pat + noise Pat What did Fido do? (bite agent Fido + bite object Pat) Fido 1 = bite agent + noise bite agent Clean-up memory An auto-associative collection of elements and statements (excluding single chunks) produced by the system. Given a noisy extracted vector such structure must be able to: either recall the most similar item stored or indicate, that no matching object had been found. We need a measure of similarity (in most models: scalar product, Hamming/Euclidean distance).

25 Distributed representations Requirements/Problems

26 Distributed representations Requirements/Problems, + - symmetric, preserving an equal portion of every component.

27 Distributed representations Requirements/Problems, + - symmetric, preserving an equal portion of every component., inverse - easy/fast to compute.

28 Distributed representations Requirements/Problems, + - symmetric, preserving an equal portion of every component., inverse - easy/fast to compute. Rules of composition and decomposition must be applicable to all elements of the domain.

29 Distributed representations Requirements/Problems, + - symmetric, preserving an equal portion of every component., inverse - easy/fast to compute. Rules of composition and decomposition must be applicable to all elements of the domain. Fixed size of vectors irrespectively of the degree of complication.

30 Distributed representations Requirements/Problems, + - symmetric, preserving an equal portion of every component., inverse - easy/fast to compute. Rules of composition and decomposition must be applicable to all elements of the domain. Fixed size of vectors irrespectively of the degree of complication. Advantages Noise tolerance, good error correction properties. Storage efficiency. The number of superimposed patterns is not fixed.

31 Previous Architectures Binary Spatter Code (Pentii Kanerva, 1997)

32 Previous Architectures Binary Spatter Code (Pentii Kanerva, 1997) Data: n-bit binary vectors of great length (n 10000)

33 Previous Architectures Binary Spatter Code (Pentii Kanerva, 1997) Data: n-bit binary vectors of great length (n 10000) Both and performed by XOR

34 Previous Architectures Binary Spatter Code (Pentii Kanerva, 1997) Data: n-bit binary vectors of great length (n 10000) Both and performed by XOR + majority rule addition (with additional random vectors to break the tie)

35 Previous Architectures Binary Spatter Code (Pentii Kanerva, 1997) Data: n-bit binary vectors of great length (n 10000) Both and performed by XOR + majority rule addition (with additional random vectors to break the tie) Similarity measure: Hamming distance

36 Previous Architectures Binary Spatter Code (Pentii Kanerva, 1997) Data: n-bit binary vectors of great length (n 10000) Both and performed by XOR + majority rule addition (with additional random vectors to break the tie) Similarity measure: Hamming distance Holographic Reduced Representation (Tony Plate, 1994, 2003)

37 Previous Architectures Binary Spatter Code (Pentii Kanerva, 1997) Data: n-bit binary vectors of great length (n 10000) Both and performed by XOR + majority rule addition (with additional random vectors to break the tie) Similarity measure: Hamming distance Holographic Reduced Representation (Tony Plate, 1994, 2003) Data: normalized n-byte vectors chosen from N(0, 1/n)

38 Previous Architectures Binary Spatter Code (Pentii Kanerva, 1997) Data: n-bit binary vectors of great length (n 10000) Both and performed by XOR + majority rule addition (with additional random vectors to break the tie) Similarity measure: Hamming distance Holographic Reduced Representation (Tony Plate, 1994, 2003) Data: normalized n-byte vectors chosen from N(0, 1/n) performed by circular convolution

39 Previous Architectures Binary Spatter Code (Pentii Kanerva, 1997) Data: n-bit binary vectors of great length (n 10000) Both and performed by XOR + majority rule addition (with additional random vectors to break the tie) Similarity measure: Hamming distance Holographic Reduced Representation (Tony Plate, 1994, 2003) Data: normalized n-byte vectors chosen from N(0, 1/n) performed by circular convolution + normalized addition, e.g. (bite + bite agent Fido + bite object Pat)/ 3

40 Previous Architectures Binary Spatter Code (Pentii Kanerva, 1997) Data: n-bit binary vectors of great length (n 10000) Both and performed by XOR + majority rule addition (with additional random vectors to break the tie) Similarity measure: Hamming distance Holographic Reduced Representation (Tony Plate, 1994, 2003) Data: normalized n-byte vectors chosen from N(0, 1/n) performed by circular convolution + normalized addition, e.g. (bite + bite agent Fido + bite object Pat)/ 3 Inverse: involution x i = x ( i)mod(n)

41 Previous Architectures Binary Spatter Code (Pentii Kanerva, 1997) Data: n-bit binary vectors of great length (n 10000) Both and performed by XOR + majority rule addition (with additional random vectors to break the tie) Similarity measure: Hamming distance Holographic Reduced Representation (Tony Plate, 1994, 2003) Data: normalized n-byte vectors chosen from N(0, 1/n) performed by circular convolution + normalized addition, e.g. (bite + bite agent Fido + bite object Pat)/ 3 Inverse: involution xi = x ( i)mod(n) ( circular correlation (i.e. circular convolution with an involution), e.g: (bite + bite agent Fido + bite object Pat)/ ) 3 biteagent

42 Previous Architectures Binary Spatter Code (Pentii Kanerva, 1997) Data: n-bit binary vectors of great length (n 10000) Both and performed by XOR + majority rule addition (with additional random vectors to break the tie) Similarity measure: Hamming distance Holographic Reduced Representation (Tony Plate, 1994, 2003) Data: normalized n-byte vectors chosen from N(0, 1/n) performed by circular convolution + normalized addition, e.g. (bite + bite agent Fido + bite object Pat)/ 3 Inverse: involution xi = x ( i)mod(n) ( circular correlation (i.e. circular convolution with an involution), e.g: (bite + bite agent Fido + bite object Pat)/ ) 3 biteagent Similarity measure: dot product

43 Binary parametrization of the geometric product Binary parametrization

44 Binary parametrization of the geometric product Binary parametrization Let {x 1,..., x n} be a string of bits and {e 1,..., e n} be base vectors. Then where e 0 k = 1. e x1...x n = e x exn n

45 Binary parametrization of the geometric product Binary parametrization Let {x 1,..., x n} be a string of bits and {e 1,..., e n} be base vectors. Then e x1...x n = e x exn n where e 0 k = 1. Example: 1 = e e 1 = e e 235 = e

46 Binary parametrization of the geometric product Binary parametrization Let {x 1,..., x n} be a string of bits and {e 1,..., e n} be base vectors. Then e x1...x n = e x exn n where e 0 k = 1. Example: 1 = e e 1 = e e 235 = e Geometric product as a projective XOR representation

47 Binary parametrization of the geometric product Binary parametrization Let {x 1,..., x n} be a string of bits and {e 1,..., e n} be base vectors. Then e x1...x n = e x exn n where e 0 k = 1. Example: 1 = e e 1 = e e 235 = e Geometric product as a projective XOR representation Example: e 12 e 1 = e e = e 1 e 2 e 1 = e 2 e 1 e 1 = e 2 = e = e ( ) (10...0) = ( 1) D e ( ) (10...0)

48 Binary parametrization of the geometric product Binary parametrization Let {x 1,..., x n} be a string of bits and {e 1,..., e n} be base vectors. Then e x1...x n = e x exn n where e 0 k = 1. Example: 1 = e e 1 = e e 235 = e Geometric product as a projective XOR representation Example: e 12 e 1 = e e = e 1 e 2 e 1 = e 2 e 1 e 1 = e 2 = e = e ( ) (10...0) = ( 1) D e ( ) (10...0) More formally, for two arbitrary strings of bits we have: B k A l e A1...A n e B1...B n = ( 1) k<l e(a1...a n) (B 1...B n)

49 Matrix representation Pauli s matrices ( 0 1 σ 1 = 1 0 ) ( 0 i, σ 2 = i 0 ) ( 1 0, σ 3 = 0 1 ).

50 Matrix representation Pauli s matrices ( 0 1 σ 1 = 1 0 ) ( 0 i, σ 2 = i 0 ) ( 1 0, σ 3 = 0 1 ). Cartan representation b 2k = σ 1 σ 1 σ 2 1 1, }{{}}{{} n k k 1 b 2k 1 = σ 1 σ 1 σ }{{}}{{} n k k 1

51 Example Representation PSmith = name Pat + sex male + age 66

52 Example Representation PSmith = name Pat + sex male + age 66 Pat = c 00100

53 Example Representation PSmith = name Pat + sex male + age 66 Pat = c = b 3

54 Example Representation PSmith = name Pat + sex male + age 66 Pat = c = b 3 = σ 1 σ 1 σ 1 σ 3 1

55 Example Representation PSmith = name Pat + sex male + age 66 Pat = c = b 3 = σ 1 σ 1 σ 1 σ 3 1 male = c = b 3 b 4 b 5 = (σ 1 σ 1 σ 1 σ 3 1)(σ 1 σ 1 σ 1 σ 2 1)(σ 1 σ 1 σ 3 1 1) = σ 1 σ 1 σ 3 ( iσ 1 ) 1

56 Example Representation PSmith = name Pat + sex male + age 66 Pat = c = b 3 = σ 1 σ 1 σ 1 σ 3 1 male = c = b 3 b 4 b 5 = (σ 1 σ 1 σ 1 σ 3 1)(σ 1 σ 1 σ 1 σ 2 1)(σ 1 σ 1 σ 3 1 1) = σ 1 σ 1 σ 3 ( iσ 1 ) 1 66 = c = b 1 b 2 = (σ 1 σ 1 σ 1 σ 1 σ 3 )(σ 1 σ 1 σ 1 σ 1 σ 2 ) = ( iσ 1 )

57 Example Representation PSmith = name Pat + sex male + age 66 Pat = c = b 3 = σ 1 σ 1 σ 1 σ 3 1 male = c = b 3 b 4 b 5 = (σ 1 σ 1 σ 1 σ 3 1)(σ 1 σ 1 σ 1 σ 2 1)(σ 1 σ 1 σ 3 1 1) = σ 1 σ 1 σ 3 ( iσ 1 ) 1 66 = c = b 1 b 2 = (σ 1 σ 1 σ 1 σ 1 σ 3 )(σ 1 σ 1 σ 1 σ 1 σ 2 ) = ( iσ 1 ) name = c = b 4 = σ 1 σ 1 σ 1 σ 2 1

58 Example Representation PSmith = name Pat + sex male + age 66 Pat = c = b 3 = σ 1 σ 1 σ 1 σ 3 1 male = c = b 3 b 4 b 5 = (σ 1 σ 1 σ 1 σ 3 1)(σ 1 σ 1 σ 1 σ 2 1)(σ 1 σ 1 σ 3 1 1) = σ 1 σ 1 σ 3 ( iσ 1 ) 1 66 = c = b 1 b 2 = (σ 1 σ 1 σ 1 σ 1 σ 3 )(σ 1 σ 1 σ 1 σ 1 σ 2 ) = ( iσ 1 ) name = c = b 4 = σ 1 σ 1 σ 1 σ 2 1 sex = c = b 1 b 2 b 3 = (σ 1 σ 1 σ 1 σ 1 σ 3 )(σ 1 σ 1 σ 1 σ 1 σ 2 )(σ 1 σ 1 σ 1 σ 3 1) = σ 1 σ 1 σ 3 1 ( iσ 1 )

59 Example Representation PSmith = name Pat + sex male + age 66 Pat = c = b 3 = σ 1 σ 1 σ 1 σ 3 1 male = c = b 3 b 4 b 5 = (σ 1 σ 1 σ 1 σ 3 1)(σ 1 σ 1 σ 1 σ 2 1)(σ 1 σ 1 σ 3 1 1) = σ 1 σ 1 σ 3 ( iσ 1 ) 1 66 = c = b 1 b 2 = (σ 1 σ 1 σ 1 σ 1 σ 3 )(σ 1 σ 1 σ 1 σ 1 σ 2 ) = ( iσ 1 ) name = c = b 4 = σ 1 σ 1 σ 1 σ 2 1 sex = c = b 1 b 2 b 3 = (σ 1 σ 1 σ 1 σ 1 σ 3 )(σ 1 σ 1 σ 1 σ 1 σ 2 )(σ 1 σ 1 σ 1 σ 3 1) = σ 1 σ 1 σ 3 1 ( iσ 1 ) age = c = b 1 b 5 = (σ 1 σ 1 σ 1 σ 1 σ 3 )(σ 1 σ 1 σ 3 1 1) = 1 1 ( iσ 2 ) σ 1 σ 3

60 Example Encoding and Decoding PSmith

61 Example Encoding and Decoding PSmith PSmith = name Pat + sex male + age 66

62 Example Encoding and Decoding PSmith PSmith = name Pat + sex male + age 66 = c c c c c c 11000

63 Example Encoding and Decoding PSmith PSmith = name Pat + sex male + age 66 = c c c c c c = c c c 01001

64 Example Encoding and Decoding PSmith PSmith = name Pat + sex male + age 66 = c c c c c c = c c c PSmith s name = PSmith name 1 = PSmith name

65 Example Encoding and Decoding PSmith PSmith = name Pat + sex male + age 66 = c c c c c c = c c c PSmith s name = PSmith name 1 = PSmith name PSmith name = ( c c c )c 00010

66 Example Encoding and Decoding PSmith PSmith = name Pat + sex male + age 66 = c c c c c c = c c c PSmith s name = PSmith name 1 = PSmith name PSmith name = ( c c c )c = c c c 01011

67 Example Encoding and Decoding PSmith PSmith = name Pat + sex male + age 66 = c c c c c c = c c c PSmith s name = PSmith name 1 = PSmith name PSmith name = ( c c c )c = c c c = Pat + noise = Pat

68 Example Clean-up Scalar product Scalar (inner) product is performed by the means of matrix trace.

69 Example Clean-up Scalar product Scalar (inner) product is performed by the means of matrix trace. ) Pat e Pat = Tr (( c c c )c ) = Tr ( 1 + c c = 32 0

70 Example Clean-up Scalar product Scalar (inner) product is performed by the means of matrix trace. ) Pat e Pat = Tr (( c c c )c ) = Tr ( 1 + c c Pat e male = 0 Pat e 66 = 0 Pat e name = 0 Pat e sex = 0 Pat e age = 0 Pat PSmith = 0 = 32 0

71 Signatures Examples

72 Signatures Examples PSmith, n = 5 PSmith, n = 7 PSmith, n = 6

73 Signatures Examples PSmith, n = 5 PSmith, n = 7 PSmith, n = 6 Signatures: LHS-signature RHS-signature

74 Signatures Dimensions

75 Signatures Dimensions 2 n 2 signatures on one of the diagonals. Each signature is of dimensions 2 n 2 2 n 2.

76 Signatures Dimensions 2 n 2 signatures on one of the diagonals. Each signature is of dimensions 2 n 2 2 n 2. Cartan representation b 2k = σ 1 σ 1 σ 2 1 1, }{{}}{{} at least n 2 k 1 b 2k 1 = σ 1 σ 1 σ }{{}}{{} at least n 2 k 1

77 Recognition tests Construction

78 Recognition tests Construction Plate (HRR) construction, e.g. eat + eat agent Mark + eat object thefish

79 Recognition tests Construction Plate (HRR) construction, e.g. eat + eat agent Mark + eat object thefish Agent-Object construction, e.g. eat agent Mark + eat object thefish

80 Recognition tests Construction Plate (HRR) construction, e.g. eat + eat agent Mark + eat object thefish Agent-Object construction, e.g. eat agent Mark + eat object thefish Vocabulary/Sentence set Similar sentences, e.g: Fido bit Pat. Fido bit PSmith. Pat fled from Fido. PSmith fled from Fido. Fido bit PSmith causing PSmith to flee from Fido. Fido bit Pat causing Pat to flee from Fido. John saw that Fido bit PSmith causing PSmith to flee from Fido. John saw that Fido bit Pat causing Pat to flee from Fido. etc... Altogether 42 atomic objects and 19 sentences.

81 Recognition tests Agent-Object construction Works better for: simple sentences and simple answers

82 Recognition tests Agent-Object construction Works better for: simple sentences and simple answers avg % of correct answers 100 Plate construction Agent-object construction 50 QUESTION: PSmith name ANSWER: Pat NUMBER OF TRIALS: n

83 Recognition tests Agent-Object construction Works better for: simple sentences and simple answers, nested sentences, from which a rather unique information is to be derived.

84 Recognition tests Agent-Object construction Works better for: simple sentences and simple answers, nested sentences, from which a rather unique information is to be derived. avg % of correct answers 100 Plate construction Agent-object construction QUESTION: (John saw that Fido bit Pat causing Pat to flee from Fido) seeagent ANSWER: John NUMBER OF TRIALS: 1000 n

85 Recognition tests Plate (HRR) construction Works better for nested sentences, from which a complex information needs to be derived. avg % of correct answers 100 Plate construction Agent-object construction 50 QUESTION: (John saw that Fido bit Pat causing Pat to flee from Fido) see object ANSWER: Fido bit Pat causing Pat to flee from Fido NUMBER OF TRIALS: 1000 n

86 Blade linearity

87 Blade linearity Problems

88 Blade linearity Problems How often does the system produce identical blades representing atomic objects?

89 Blade linearity Problems How often does the system produce identical blades representing atomic objects? How often does the system produce identical sentence chunks from different blades?

90 Blade linearity Problems How often does the system produce identical blades representing atomic objects? How often does the system produce identical sentence chunks from different blades? How do the two above problems affect the number of (pseudo-) correct answers?

91 Blade linearity Problems How often does the system produce identical blades representing atomic objects? How often does the system produce identical sentence chunks from different blades? How do the two above problems affect the number of (pseudo-) correct answers? Assumption: ideal conditions

92 Blade linearity Problems How often does the system produce identical blades representing atomic objects? How often does the system produce identical sentence chunks from different blades? How do the two above problems affect the number of (pseudo-) correct answers? Assumption: ideal conditions No two chunks of a sentence at any time are identical (up to a constant).

93 Blade linearity Problems How often does the system produce identical blades representing atomic objects? How often does the system produce identical sentence chunks from different blades? How do the two above problems affect the number of (pseudo-) correct answers? Assumption: ideal conditions No two chunks of a sentence at any time are identical (up to a constant). A C = 0 A and C share a common blade

94 Blade linearity One meaningful blade, L > 0 noisy blades avg number of probings giving a nonzero matrix trace 15 test results estimates 10 QUESTION: (Fido bit Pat) bite object ANSWER: Pat NUMBER OF TRIALS: MEANINGFUL/NOISY BLADES: 1 / n

95 Blade linearity K > 1 meaningful blades, L > 0 noisy blades avg number of probings giving a nonzero matrix trace 15 test results estimates 10 QUESTION: (Fido bit PSmith) bite object ANSWER: PSmith= name Pat + sex male + age 66 SIMILAR ANSWER: DogFido= name Fido + sex male NUMBER OF TRIALS: 1000 MEANINGFUL/NOISY BLADES: 3/ n

96 What s next? Issues

97 What s next? Issues Decide on construction (something between Plate and Agent-Object?).

98 What s next? Issues Decide on construction (something between Plate and Agent-Object?). Tests for greater n.

99 What s next? Issues Decide on construction (something between Plate and Agent-Object?). Tests for greater n. Scaling.

100 References M. Czachor, D. Aerts and B. De Moor. On geometric-algebra representation of binary spatter codes. preprint arxiv:cs/ [cs.ai] M. Czachor, D. Aerts and B. De Moor. Geometric Analogue of Holographic Reduced Representation. preprint arxiv: G. E. Hinton, J. L. McClelland and D. E. Rumelhart. Distributed representations. Parallel distributed processing: Explorations in the microstructure of cognition. vol. 1, The MIT Press, Cambridge, MA, P. Kanerva. Binary spatter codes of ordered k-tuples. Artificial Neural Networks ICANN Proceedings, Lecture Notes in Computer Science vol. 1112, pp , C. von der Malsburg et al. (Eds.). Springer, Berlin, P. Kanerva. Fully distributed representation. Proc Real World Computing Symposium (RWC97, Tokyo), pp Real World Computing Partnership, Tsukuba-City, Japan, T. Plate. Holographic Reduced Representation: Distributed Representation for Cognitive Structures. CSLI Publications, Stanford, 2003.