A Quantum-inspired version of the Nearest Mean Classifier

Transcription

1 A Quantum-inspired version of the Nearest Mean Classifier (with Roberto Giuntini and Ranjith Venkatrama) University of Cagliari November 6th-8th, Verona Quantum Techniques in Machine Learning - QTML 2017

2 A foundational question Can Quantum Theory help classical machine learning (by using classical computer only)? A naive example.

3 Training set, Class, Pattern, Feature Nearest Mean Classifier (NMC) Standard (supervised) Classification scenario Let us consider (as a simple example) two disjoint sets A and B of different objects (say cats and dogs). During the training process, we take n objects from the set A and m objects from the set B. Let C a A and C b B. We can measure two (or more) features of each object a i C a and b i C b (for istance the weight and the lenght of the tail). We say that C a and C b are classes and the objects a i and b i are patterns that are characterized by their features. We write, for example, a i = {x 1, x 2 }, where x 1 and x 2 are the weight and the lenght of the tail of the cat a i, respectively.

4 Training set, Class, Pattern, Feature Nearest Mean Classifier (NMC) Nearest Mean Classifier (NMC) Let us consider the classes C a = {a 1,..., a n } and C b = {b 1,..., b m }, with a i and b i belonging to the training set and an arbitrary pattern c i = {x 1, x 2 } belonging to the test set. The goal is to establish whether is more probably that c i A or c i B. We - only - consider the centroids a and b of C a and C b and the euclidean distances Ed(c i, a ) and Ed(c i, b ). Hence, if Ed(c i, a ) Ed(c i, b ) then (is more probabily that) c i B; otherwise c i A.

5 How to encode a Pattern as a Density operator Normalized Trace Distance All we need in order to provide a Quantum representation of NMC are: a suitable encoding from patterns to quantum objects a quantum counterpart for the Euclidean distance a quantum counterpart for the centroid

6 How to encode a Pattern as a Density operator Normalized Trace Distance Encoding. An Example: Stereographic encoding It is possible to map the pattern a = (x, y) onto the surface of a radius one sphere by the stereographic projection: 2x (x, y) ( x 2 + y 2 + 1, 2y x 2 + y 2 + 1, x 2 + y 2 1 x 2 + y ). By placing the Bloch components: 2x r 1 = x 2 +y 2 +1 ; r 2y 2 = x 2 +y 2 +1 ; r 3 = x 2 +y 2 1 we obtain: x 2 +y 2 +1 ρ a = 1 ( ) 1 + r3 r 1 ir 2 = 2 r 1 + ir 2 1 r 3 1 x 2 + y ( ) x 2 + y 2 x iy. x + iy 1

7 How to encode a Pattern as a Density operator Normalized Trace Distance Example Let us consider the pattern a = {1, 3}. Its corresponding Density pattern ρ a, is: ρ a = 1 11 ( ) i 1 + 3i 1 We call ρ a Density Pattern.

8 Basic Notions How to encode a Pattern as a Density operator Normalized Trace Distance Moon Dataset Figure : Classical Patterns Figure : Density Patterns

9 How to encode a Pattern as a Density operator Normalized Trace Distance Another Example: Projective encoding v (x, y) ( x v, y ) ( x, ȳ) v ψ v = x 0 + ȳ 1 ρ v = ψ v ψ v...and many others.

10 How to encode a Pattern as a Density operator Normalized Trace Distance Distance. Preservation of the Order Let a = {x a, y a } b = {x b, y b } and c = {x c, y c } be three arbitrary patterns and let ρ i be the density pattern associated to the pattern i. If Ed(a, b) Ed(b, c) (where Ed is the Euclidian distance), is it possible to define a Quantum distance such that Qd(ρ a, ρ b ) Qd(ρ b, ρ c )?

11 How to encode a Pattern as a Density operator Normalized Trace Distance Normalized Trace Distance Let us consider two patterns a = {x a, y a } and b = {x b, y b }. ( Let ρ a = ra3 r a1 ir a2 2 r a1 + ir a2 1 r a3 associated to a; similarly for b. ) the density pattern 2 Let place K = (1 ra3 and let we define the normalized )(1 r b3 ) trace distance as: K Td(ρ a, ρ b ), where Td is the usual Trace distance. It is straightforward to show that Ed(a, b) = K Td(ρ a, ρ b ).

12 How to encode a Pattern as a Density operator Normalized Trace Distance At this stage let s keep the definition of centroid unchanged. Classification Hence, given a and b as the centroids of C a and C b respectively, if K Td(ρ x, ρ a ) K Td(ρ x, ρ b ) then x B; otherwise x A. Similarly to the classical case.

13 How to encode a Pattern as a Density operator Normalized Trace Distance In this way we have introduced just a translation of the standard NMC in terms of quantum objects (with the pontential implicit benefits carried out by a Quantum Computer). But this approach turns out to be convenient also on a Classical Computer.

14 Quantum Centroid Comparison Now we introduce a new definition of Quantum Centroid that has not any counterpart in the classical framework. Quantum Centroid Given a dataset {P 1,..., P n }, let us consider the respective set of density patterns {ρ 1,..., ρ n }. The Quantum Centroid is defined as: ρ QC = 1 n n ρ i. i=1 S. Gambs, Quantum classification, arxiv: v2.

15 Quantum Centroid Comparison Observations Some observation: The QC ρ QC is not a pure state and it has not any counterpart in the set of classical pattern in R n ; In contrast to the Classical Centroid, the QC is "sensitive" to the distribution of the patterns.

16 Quantum Centroid Comparison We provide a comparison between the NMC and the "quantum" classification process based on Density Patterns and Quantum Centroids by involving different kinds of standard (artificial) datasets on a Classical Computer. We compare the Error E (E = #CorrectClassifiedPatterns #PatternsOfTheDataset ) and the reliability (in terms of the Cohen s constant k) for both classifiers.

17 Quantum Centroid Comparison Gaussian Dataset Gaussian Dataset: 200 Patterns allocated in two Classes. Figure : Gaussian Dataset Figure : NMC Figure : Quantum Classifier

18 Quantum Centroid Comparison Table : Gaussian Dataset E E1 E2 Pr k TPR FPR NMC QC By randomly dividing the dataset in a training set (80%) and in a test set (20%), the average over 100 experiments gives: NMC Error = ± 6.79; Q Error = ± 6.09.

19 Quantum Centroid Comparison A remark Even if the error of the Quantum Classifier is lower than the Error of the NMC, there are some patterns that are correctly classified by the NMC but not by the Quantum Classifier. Hence, it makes sense to consider a "merging" of the NMC and the Quantum Classifier.

20 Basic Notions Quantum Centroid Comparison Gaussian Dataset Figure : Gaussian Dataset Figure : NMC Figure : Quantum Classifier Figure : NMC & Quantum Classifier

21 Quantum Centroid Comparison Table : Gaussian Dataset E E1 E2 Pr k TPR FPR NMC QC NMC-QC

22 Quantum Centroid Comparison Moon Dataset Moon Dataset: 200 patterns allocated in two Classes. Figure : Moon Dataset Figure : NMC Figure : Quantum Classifier Figure : NMC & Quantum Classifier

23 Quantum Centroid Comparison Table : Moon Dataset E E1 E2 Pr k TPR FPR NMC QC By randomly dividing the dataset in a training set (80%) and in a test set (20%), the average over 100 experiments gives: NMC Error = ± 6.32; Q Error = ± 5.46.

24 Basic Notions Quantum Centroid Comparison Banana Dataset Banana Dataset: 5300 patterns; 2376 belonging to the first Class and 2924 to the second Class. Figure : Banana Dataset Figure : NMC Figure : Quantum Classifier Figure : NMC & Quantum Classifier

25 Quantum Centroid Comparison Table : Banana Dataset E E1 E2 Pr k TPR FPR NMC QC NMC-QC By randomly dividing the dataset in a training set (80%) and in a test set (20%), the average over 100 experiments gives: NMC Error = ± 1.74; Q Error = ± 1.21.

26 Basic Notions Quantum Centroid Comparison 3Gaussian Dataset 3Gaussian Dataset: 450 Patterns allocated in three Classes. Figure : 3Gaussian Dataset Figure : NMC Figure : Quantum Classifier Figure : NMC & Quantum Classifier

27 Quantum Centroid Comparison Here we randomly divide the dataset in a Training set (80% of the patterns) and a Test set (20% of the patterns). We calculate the average over 100 runs for each experiments. A full comparison

28 The Quantum Centroid is not invariant under rescaling the "Quantum" Classifier is not invariant under rescaling! Is it an Embarrasment or is it an Asset? The Error is dependent on both the rescaling of the Patterns and the different encoding. We show how the Error changes by changing the rescaling (it can be shown for two different encodings).

29 Rescaling How the Error of the Quantum Classifier changes by ranging the value of the rescaling. If we know, in principle, the kind of distribution of a given phenomena, we can choose ad hoc the suitable rescaling in order to further improve the accuracy of the classification.

30 Summarizing the procedure 1. Start from classical data 2. Suitable rescaling (if we know in principle the kind of distribution) 3. Encoding (from classical patterns to density patterns) 4. Classification (using a classical computer) 5. Decoding (from density patterns to classical patterns) 6. Merging with the classical result Positive answer to the foundational question.

31 The choise of the "best" Encoding (and/or the best Rescaling) is mostly empirical and it is stritcly dependent on the Dataset. A complete theoretical investigation is still missing!!! A comparison with more performant classifiers (Linear Discriminant Analysis, Quadratic Discriminant Analysis...) also in case of unsupervised or semi-supervised classification could be investigated. (The NMC and the Quantum Classifier are based on the concepts of centroid and distance only but more sofisticated classifiers can be replaced in a quantum framework exploiting superposition principle, entanglement, majorization etc...).

32 Basic Notions From artificial to real datasets As further developments, it will be checked whether the Quantum Classifier could bring some benefit for practical implementations, such as Figure : Handwriting Figure : Fingerprint Recognition Figure : Face Recognition Figure : Biomedical Imaging

33 thank you G. Sergioli, E. Santucci, L. Didaci, J.A. Miskczak, R. Giuntini, A Quantum-inspired version of the Nearest Mean Classifier, Soft Computing (2016). G. Sergioli, G.M. Bosyk, E. Santucci, R. Giuntini, A Quantum-inspired version of the classification problem, International Journal of Theoretical Physics (2017). F. Holik, G. Sergioli, H. Freytes, A. Plastino, Pattern Recognition in non-kolmogorovian Structures, Foundation of Science (2017). E. Santucci, G. Sergioli, Classification Problem in a Quantum Framework, Proceedings of the Vaxjo Conference Quantum and Beyond (2017).