Cellular Automata Evolution for Pattern Recognition

Similar documents
Error Correcting Capability of Cellular Automata Based Associative Memory

Integer weight training by differential evolution algorithms

Convolutional Associative Memory: FIR Filter Model of Synapse

Data Mining. Preamble: Control Application. Industrial Researcher s Approach. Practitioner s Approach. Example. Example. Goal: Maintain T ~Td

Logic BIST. Sungho Kang Yonsei University

Switching Neural Networks: A New Connectionist Model for Classification

Machine Learning. Neural Networks

Optimization of 1D and 2D Cellular Automata for Pseudo Random Number Generator.

Discrete Tranformation of Output in Cellular Automata

Theory of Additive Cellular Automata

CSCI 1590 Intro to Computational Complexity

Keywords- Source coding, Huffman encoding, Artificial neural network, Multilayer perceptron, Backpropagation algorithm

Computational Intelligence Lecture 3: Simple Neural Networks for Pattern Classification

CS:4420 Artificial Intelligence

Using a Hopfield Network: A Nuts and Bolts Approach

The Power of Extra Analog Neuron. Institute of Computer Science Academy of Sciences of the Czech Republic

Intelligent Modular Neural Network for Dynamic System Parameter Estimation

Principles of Pattern Recognition. C. A. Murthy Machine Intelligence Unit Indian Statistical Institute Kolkata

Probabilistic Analysis of Cellular Automata Rules and its Application in Pseudo Random Pattern Generation

Bayesian Networks Inference with Probabilistic Graphical Models

Research Article Characterization of the Evolution of Nonlinear Uniform Cellular Automata in the Light of Deviant States

Outline. EECS Components and Design Techniques for Digital Systems. Lec 18 Error Coding. In the real world. Our beautiful digital world.

International Journal of Combined Research & Development (IJCRD) eissn: x;pissn: Volume: 7; Issue: 7; July -2018

Last update: October 26, Neural networks. CMSC 421: Section Dana Nau

Radial View: Observing Fuzzy Cellular Automata with a New Visualization Method

Hopfield Network Recurrent Netorks

Implementation of Lossless Huffman Coding: Image compression using K-Means algorithm and comparison vs. Random numbers and Message source

Neural Networks. Chapter 18, Section 7. TB Artificial Intelligence. Slides from AIMA 1/ 21

Neural networks. Chapter 19, Sections 1 5 1

Determine the size of an instance of the minimum spanning tree problem.

Artificial Neural Networks Examination, June 2005

Artificial Neural Networks Examination, March 2004

Artificial Intelligence Hopfield Networks

Neural Networks. Mark van Rossum. January 15, School of Informatics, University of Edinburgh 1 / 28

Part 8: Neural Networks

Reification of Boolean Logic

CMSC 421: Neural Computation. Applications of Neural Networks

DRAFT. Algebraic computation models. Chapter 14

ARTIFICIAL NEURAL NETWORK PART I HANIEH BORHANAZAD

Text Mining. Dr. Yanjun Li. Associate Professor. Department of Computer and Information Sciences Fordham University

INTRODUCTION TO ARTIFICIAL INTELLIGENCE

Image Encryption and Decryption Algorithm Using Two Dimensional Cellular Automata Rules In Cryptography

Artificial Intelligence (AI) Common AI Methods. Training. Signals to Perceptrons. Artificial Neural Networks (ANN) Artificial Intelligence

Neural networks. Chapter 20, Section 5 1

Computational Intelligence Lecture 6: Associative Memory

AN INTRODUCTION TO NEURAL NETWORKS. Scott Kuindersma November 12, 2009

A Novel Activity Detection Method

CS 4700: Foundations of Artificial Intelligence

Sorting Network Development Using Cellular Automata

Mechanisms of Emergent Computation in Cellular Automata

EEE 241: Linear Systems

Lecture 7 Artificial neural networks: Supervised learning

Learning and Memory in Neural Networks

Artificial Neural Networks. Q550: Models in Cognitive Science Lecture 5

Cellular Automata. ,C ) (t ) ,..., C i +[ K / 2] Cellular Automata. x > N : C x ! N. = C x. x < 1: C x. = C N+ x.

Artificial Neural Networks Examination, June 2004

Online Dictionary Learning with Group Structure Inducing Norms

Non-linear Measure Based Process Monitoring and Fault Diagnosis

Weight Quantization for Multi-layer Perceptrons Using Soft Weight Sharing

Chapter 1. Introduction

Modeling High-Dimensional Discrete Data with Multi-Layer Neural Networks

Designing Cellular Automata Structures using Quantum-dot Cellular Automata

Sections 18.6 and 18.7 Analysis of Artificial Neural Networks

Artificial Neural Network Method of Rock Mass Blastability Classification

Coexistence of Dynamics for Two- Dimensional Cellular Automata

Design and Implementation of Carry Adders Using Adiabatic and Reversible Logic Gates

One-Dimensional Linear Hybrid Cellular Automata: Their Synthesis, Properties and Applications to Digital Circuits Testing

Support Vector Machines (SVM) in bioinformatics. Day 1: Introduction to SVM

BOOLEAN ALGEBRA INTRODUCTION SUBSETS

Bioinformatics Chapter 1. Introduction

Classification of Random Boolean Networks

A Genetic Algorithm with Expansion and Exploration Operators for the Maximum Satisfiability Problem

Application of hopfield network in improvement of fingerprint recognition process Mahmoud Alborzi 1, Abbas Toloie- Eshlaghy 1 and Dena Bazazian 2

ECE 407 Computer Aided Design for Electronic Systems. Simulation. Instructor: Maria K. Michael. Overview

A Posteriori Corrections to Classification Methods.

Evolutionary Functional Link Interval Type-2 Fuzzy Neural System for Exchange Rate Prediction

Decision Tree Learning

Neural Networks Lecture 2:Single Layer Classifiers

Sample questions for Fundamentals of Machine Learning 2018

Modern Information Retrieval

CS6901: review of Theory of Computation and Algorithms

Introduction to Neural Networks

Stream Ciphers. Çetin Kaya Koç Winter / 20

NCU EE -- DSP VLSI Design. Tsung-Han Tsai 1

Neural Networks and Fuzzy Logic Rajendra Dept.of CSE ASCET

ARTIFICIAL NEURAL NETWORKS گروه مطالعاتي 17 بهار 92

Chapter 0 Introduction. Fourth Academic Year/ Elective Course Electrical Engineering Department College of Engineering University of Salahaddin

biologically-inspired computing lecture 18

Inadmissible Class of Boolean Functions under Stuck-at Faults

SP-CNN: A Scalable and Programmable CNN-based Accelerator. Dilan Manatunga Dr. Hyesoon Kim Dr. Saibal Mukhopadhyay

Iterative Laplacian Score for Feature Selection

Bounded Approximation Algorithms

Fundamentals of Digital Design

CMSC 422 Introduction to Machine Learning Lecture 4 Geometry and Nearest Neighbors. Furong Huang /

SUPERVISED LEARNING: INTRODUCTION TO CLASSIFICATION

From statistics to data science. BAE 815 (Fall 2017) Dr. Zifei Liu

Fault Collapsing in Digital Circuits Using Fast Fault Dominance and Equivalence Analysis with SSBDDs

Genetic Algorithms: Basic Principles and Applications

DIAGNOSIS OF FAULT IN TESTABLE REVERSIBLE SEQUENTIAL CIRCUITS USING MULTIPLEXER CONSERVATIVE QUANTUM DOT CELLULAR AUTOMATA

On Detecting Multiple Faults in Baseline Interconnection Networks

Transcription:

Cellular Automata Evolution for Pattern Recognition Pradipta Maji Center for Soft Computing Research Indian Statistical Institute, Kolkata, 700 108, INDIA Under the supervision of Prof. P Pal Chaudhuri Prof. Debesh K Das Professor Emeritus Professor Dept. of Comp. Sci. & Tech. Dept. of Comp. Sci. & Engg. Bengal Engineering College (DU), Shibpur, INDIA Jadavpur University, INDIA

Introduction Cellular Automata (CA) promising research area Artificial Intelligence (AI) Artificial Life (ALife) Considerable research in modeling tool image processing language recognition pattern recognition VLSI testing Cellular Automata (CA) learns association from a set of examples apply this knowledge-base to handle unseen cases such associations effective for classifying patterns

Contribution of the Thesis Analysis and synthesis of linear boolean CA (MACA) CA with only XOR logic application of MACA in pattern recognition data mining image compression fault diagnosis of electronic circuit Analysis and synthesis of non-linear boolean CA (GMACA) CA with all possible logic application of non-linear CA in pattern recognition Analysis and synthesis of fuzzy CA (FMACA) CA with fuzzy logic application of fuzzy CA in pattern recognition

Cellular Automata (CA) A special type of computing model 50 s - J Von Neumann 80 s - S. Wolfram A CA displays three basic characteristics Simplicity: Basic unit of CA cell is simple Vast parallelism: CA achieves parallelism on a scale larger than massively parallel computers Locality: CA characterized by local connectivity of its cell all interactions take place on a purely local basis a cell can only communicate with its neighboring cells interconnection links usually carry only a small amount of information no cell has a global view of the entire system

Cellular Automata (CA) A computational model with discrete cells updated synchronously Uniform CA, hybrid / non-uniform CA, null boundary CA, periodic boundary CA Each cell can have 256 different rules.. 2 - state 3-neighborhood CA cell Clock Input output 0/1 From left neighbor Combinati onal Logic From right neighbor

For 2nd Cell Rule 230 PS NS 111 1 110 1 101 1 100 0 011 0 010 1 001 1 000 0 CA State Transition 0 0 1 1 98 230 226 107 0 1 1 1 98 230 226 107 0 0 1 0 3 7 2

Different Types of CA Linear CA Based on XOR logic Total 7 rules (60, 90, 102, 150, 170, 204, 240) Can be expressed through matrix (T), characteristic polynomial Next state of the CA cell P(t+1) = T. P(t) 60 102 150 204 Additive CA Based on XOR and XNOR logic Total 14 rules (linear rules + 195,165,153,105,85,51,15) Can be expressed through matrix, inversion vector, characteristic polynomial The next state of the CA cell P(t+1) = T. P(t) + F T = 1 0 0 0 0 1 1 0 0 1 1 1 0 0 0 1 60 153 105 204 F = 0 1 1 0

Additive Cellular Automata XOR Logic XNOR Logic Rule 60 : q I (t+1) = q I-1 (t) q I (t) Rule 90 : q I (t+1) = q I-1 (t) q I+1 (t) Rule 195 : q I (t+1) = q I-1 (t) q I (t) Rule 165 : q I (t+1) = q I-1 (t) q I+1 (t) Rule 102 : q I (t+1) = q I (t) q I+1 (t) Rule 153 : q I (t+1) = q I (t) q I+1 (t) Rule 150 : q I (t+1) = q I-1 (t) q I (t) q I-1 (t) Rule 105 : q I (t+1) = q I-1 (t) q I (t) q I-1 (t) Rule 170 : q I (t+1) = q I-1 (t) Rule 204 : q I (t+1) = q I (t) Rule 85 : q I (t+1) = q I-1 (t) Rule 51 : q I (t+1) = q I (t) Rule 240 : q I (t+1) = q I+1 (t) Rule 15 : q I (t+1) = q I+1 (t)

CA - State Transition Diagram 0 6 9 15 Group CA 13 7 12 3 14 11 5 2 8 1 4 10 2 5 7 8 13 10 3 4 6 9 12 11 15 14 Non-group Cellular Automata Linear Non-linear Fuzzy 0 1 Non-group CA Associative Memory MACA GMACA FMACA Perform pattern recognition task

Pattern Recognition Pattern Recognition/Classification most important foundation stone of knowledge extraction methodology demands automatic identification of patterns of interest (objects, images) from its background (shapes, forms, outlines, etc) conventional approach machine compares given input pattern with each of stored patterns identifies the closest match time to recognize the closest match O(k) recognition slow Associative Memory Entire state space - divided Transient into some pivotal points Transient Transient 1. MACA (linear) 2. GMACA (non-linear) 3. FMACA (fuzzy) States close to pivot - associated with that pivot Time to recognize a pattern - Independent of number of stored patterns

Multiple Attractor CA (MACA) Employs linear CA rules State with self loop attractor Transient states and attractor form attractor basin Behaves as an associative memory Forms natural clusters 10001 01001 10010 10011 10000 11000 01000 01010 11011 01011 00001 00000 11001 00010 00011 11010 10100 10101 10110 10111 01100 11100 01101 01110 11111 01111 00101 00100 11101 00110 00111 11110

Multiple Attractor CA (MACA) Next state: P(t+1) = T. P(t) Characteristic Polynomial: X (n-m) (1+X) m where m=log 2 (k) n denotes number of CA cell k denotes number of attractor basins Depth d of MACA number of edges between a non-reachable state and an attractor state Attractor of a basin: P(t+d) = T d P(t) m-bit positions pseudo-exhaustive: extract PEF (pseudo-exhaustive field) from attractor state Problem: Complexity of identification of attractor basin is O(n 3 ) Exponential search space Redundant solutions

Chromosome of Genetic Algorithm T = 1 1 0 0 0 1 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 1 0 Rule vector 102 60 204 204 240 Matrix 1 1 1 1 0.. 1 0 Characteristic polynomial x 3 (1+x) 2 Elementary divisors x 3 (1+x) 2 x 2 (1+x) (1+x) x x 2 (1+x) ( 1+x) x

Dependency Vector/Dependency String T = 1 1 0 0 0 1 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 1 0 T1 = 1 1 0 1 1 0 0 0 1 Characteristic polynomial x 2 (1+x) Characteristic polynomial x 3 (1+x) 2 T2 = 1 0 1 0 Characteristic polynomial x(1+x) Matrix T is obtained from T1 and T2 by Block Diagonal Method

Dependency Vector/Dependency String T d = 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 1 0 Dependency Vector DV1 = < 0 0 1 > Dependency String DS = < 0 0 1 > < 1 0 > Dependency String DS = < 0 0 1 2 0 > T1 d = T2 d = 0 0 0 0 0 0 0 0 1 1 0 1 0 0 0 1 2 0 Dependency Vector DV1 = < 1 0 >

Dependency Vector/Dependency String Zero basin of T1 + Zero basin of T2 ----- Zero basin of T1 + Non-zero basin of T2 --- Non-zero basin of T1 + Zero basin of T2 ---- Non-zero basin of T1 + Non-zero basin of T2 --- PEF Bits 0 0 0 1 1 0 1 1 DV1 contributes 1 st PEF Bits DV2 contributes 2 nd PEF Bits PEF = [PEF1] [PEF2] = [DS.P] = [DV1.P1] [DV2.P2] P = [ 1 1 1 1 1 ] DS = [ 0 0 1 2 0 ] PEF = [PEF1][PEF2] = [<0 0 1><1 1 1>][<1 0><1 1>] = 1 1

Matrix/Rule from Dependency String 0 in DV T1 = [ 0 ] 1 in DV T2 = [ 1 ] 11 in DV T3 = [T1] 1 0 0 101 in DV T4 = DS = < 1 0 1 1 2 2 2> [T1] 1 0 0 1 1 0 1 1 DV = < 1 0 1 1 > DV = < 1 1 1 > 1 1 0 0 0 0 0 0 1 1 0 0 0 0 T1 = 1 1 0 0 0 1 1 0 0 1 1 1 T2 = 1 1 0 0 0 1 0 0 0 T = 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 CA Rule Vector <102, 102, 150, 0, 102, 170, 0> 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0

Dependency Vector/Dependency String Characteristic polynomial x 4 (1+x) T1 = 0 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 0 T2 = 0 1 0 0 0 0 0 1 0 0 0 1 1 1 0 0 0 1 0 0 0 0 0 1 0 T3 = 0 1 0 0 0 0 0 1 0 0 0 1 1 1 0 0 0 1 0 0 0 0 0 0 0 <170, 0, 150, 0, 240> <170, 0, 150, 0, 240> <170, 0, 150, 0, 240> Dependency Vector <0 1 1 1 0> Identification of attractor basins in O(n) Reduction of search space

Image Compression Block diagram of codebook generation scheme Training Images Spatial Domain High Compression ratio Acceptable image quality 16 X 16 Set 8 X 8 Set 4 X 4 Set Applications - Human Portraits TSVQ 16 X 16 Codebook 8 X 8 Codebook 4 X 4 Codebook

Tree-Structured Vector Quantization N X N Set S1, S2, S3, S4 Cluster 1 Cluster 2 S1, S2 S3, S4 Centroid 1 Centroid 2 S1 S2 S3 S4 Clusters and centroids generation using Tree-Structured Vector Quantization (TSVQ) Logical structure of multi-class classifier equivalent to PTSVQ

MACA Based Two Stage Classifier Input Layer Classifier 1 Hidden Layer Classifier 2 Output Layer Classifier 1: n-bit DS consists of m DVs Classifier 2: m-bit DV No of Bits (n) Value of PEF (m) Memory Size Ratio Software Hardware 100 5 0.795 0.942 15 0.003 0.010 MSR (software) = (n+m) / (n+2 m ) 200 5 15 0.884 0.006 0.969 0.019 MSR (hardware) = (3n+3m-4) / (3n-2+2 m ) 300 5 15 0.919 0.009 0.979 0.028

Image Compression Original Decompressed Execution Time (in milli seconds) Block Size Full Search TSVQ CA 4 X 4 0.0121 0.00824 0.00562 compression 96.43%, PSNR 32.81 8 X 8 0.0473 0.03312 0.01367 16 X 16 0.1941 0.13192 0.04102 compression 95.66%, PSNR 34.27 High compression Acceptable image quality Higher speed

MACA based Tree-Structured Classifier Selection of MACA: Diversity of i th attractor basin (node): M i = max{n ij } / j N ij where N ij - number of tuples of class j covered by i th attractor basin M i 1, i th attractor indicates class j for which N ij is maximum Figure of Merit: FM = 1/k i M i where k denotes number of attractor basins MACA 1 MACA 2 II IV MACA 3 MACA 4 I III IV I II III I II IV

Fault Diagnosis of Digital Circuit Fault Injection Diagnosis of an example CUT EC Set of Test Vect ors Module 1 Module 2 CUT EC S A Signature Set Pattern Classifier MACA Fault Injection CUT (n,p) C1908(25,1801) # Partition 6 MACA 98.83 Dictionary 96.03 Memory 0.9106 C6288(32,7648) 6 99.72 99.33 0.2104 C7552(108,7053) 6 98.96 98.81 0.2238 S4863(16,4123) 10 92.43 89.63 0.7504 S3271(14,2585) 8 97.05 79.54 0.9308 S6669(55,6358) 10 99.94 95.12 0.4426

Fault Diagnosis of Analog Circuit OTA1 OTA2 Component OTA1 # Samples 8970 Detected 8962 Not detected 8 SR 99.91 V in C1 V out C2 OTA2 C1 5430 5430 5424 5421 6 9 99.88 99.83 C2 5610 5603 7 99.87 Component # Samples Detected Not detected SR OTA3 OTA1 OTA2 OTA3 7201 4321 4441 7193 4306 4436 8 15 5 99.89 99.65 99.88 V in OTA1 X1 OTA2 X2 C1 C2 4297 4321 4181 4219 116 102 97.30 97.64 X1: Output of BPF X2: Output of LPF

Performance on STATLOG Dataset Classification Accuracy (%) Memory Overhead (Kbyte) Dataset Bayesian C4.5 MLP MACA Bayesian C4.5 MLP MACA Australian 83.4 85.8 84.7 86.5 19 37.85 1.93 8.04 Diabetes 72.9 74.2 75.3 75.9 14 27.15 0.44 9.96 DNA 90.3 93.3 91.4 87.9 1000 1067.96 37.22 50.86 German 66.8 67.4 67.1 74.6 49 99.31 13.64 17.42 Heart 80.1 79.3 80.7 86.6 9.8 19.46 1.52 3.17 Satimage 85.4 85.2 86.2 77.5 669 709.72 11.33 222.74 Shuttle 99.9 99.9 99.9 94.1 1500 1513.57 0.71 55.17 Letter 87.4 86.6 67.2 84.4 766 1299.28 2.57 354.11 Vehicle 72.9 68.5 79.3 78.7 47.02 72.14 1.32 29.83 Segment 96.7 94.6 94.4 89.5 121.89 370.42 2.74 24.51

MACATree VS C4.5 on Statlog Dataset STATLOG Dataset Classifn. Accuracy C4.5 MACA Memory Overhead C4.5 MACA No of Nodes C4.5 MACA Retrieval Time(ms) C4.5 MACA Australian 85.8 86.5 37.85 8.04 35 27.3 461 4 Diabetes 74.2 75.9 27.15 9.96 39 34.2 1739 10 DNA 93.3 87.9 1067.96 50.86 127 124.3 655 494 German 67.4 74.6 99.31 17.42 134 49.8 1967 19 Heart 79.3 86.6 19.46 3.17 33 8.6 1819 17 Satimage 85.2 77.5 709.72 222.74 433 199.2 2255 4943 Shuttle 99.9 94.1 1513.57 55.17 49 47.2 3985 6633 Letter 86.6 84.4 1299.28 354.11 2107 524.6 14352 5633 Vehicle 68.5 78.7 72.14 29.83 139 50.8 1963 28 Segment 94.6 89.5 370.42 24.51 82 39.7 2508 58 Comparable classification accuracy Low memory overhead Lesser number of intermediate nodes Lesser retrieval time

Conclusion Advantages: Explore computational capability of MACA Introduction of Dependency Vector (DV)/String (DS) to characterize MACA Reduction of complexity to identify attractor basins from O(n 3 ) to O(n) Elegant evolutionary algorithm combination of DV/DS and GA MACA based tree-structured pattern classifier Application of MACA in Classification image compression fault diagnosis of electronic circuits Codon to amino acid mapping, S-box of AES Problems: Linear MACA employs only XOR logic, functionally incomplete Distribution of each attractor basin is even Can handle only binary patterns Solutions: Nonlinear MACA (GMACA) Fuzzy MACA (FMACA)

Generalized MACA (GMACA) Employs non-linear hybrid rules with all possible logic Cycle or attractor length greater than 1 Can perform pattern recognition task Behaves as an associative memory 0100 1000 1101 0111 1010 0001 0011 1100 1001 0101 0010 Rule vector: <202,168,218,42> 1011 0110 1110 0000 P1 attractor-1 P2 attractor-2 1111

Basins of Attraction (Theoretical) n = 50 Error correcting capability at single bit noise k = 10 Error correcting capability at multiple bit noise

Distribution of CA Rule (Theoretical) Degree of Homogeneity DH = 1- r/4 where r = number of 1 s of a rule More homogeneous less probability of occurrence

Synthesis of GMACA Phase I: Random Generation of a directed sub-graph Phase II: State transition table from sub-graph Phase III: GMACA rule vector from State transition table For 2nd Cell:- 111 1 110 1 101 1 100 0 011 1 010 0 001 0 000 0 Rule 232 Basin 2 0100 1000 0001 Basin 1 0010 0000 g1 Present State 1110 1011 1101 0111 1111 Basin Next State 0111 0111 0111 1111 1111 Present State 1110 1011 1101 Basin 2 0111 1111 Next State 1 0100 0001 1000 0001 0001 0000 0000 0000 0010 0000 g2

Resolution of Collision: Genetic Algorithm if n 0 = n 1 ; next state is either `0 or `1 if n 0 > n 1 ; next state is `0 if n 0 < n 1 ; next state is `1 where n 0 = Occurrence of state `0 for a configuration n 1 = Occurrence of state `1 g1 g2.. gk Example chromosome format each g x a basin of a pattern P x k numbers of genes in a chromosome Each gene - a single cycle directed sub-graph with p number of nodes, where p = 1 + n

Maximum Permissible Noise/Height Minimum value of maximum permissible height h max = 2 Minimum value of maximum permissible noise r max = 1

Performance Analysis of GMACA Higher memorizing capacity than Hopfield network Cost of computation is constant depends on transient length of CA

Basins of Attraction (Experimental) n = 50 k = 10

Distribution of CA Rule (Experimental)

Conclusion Advantages: Explore computational capability of non-linear MACA Characterization of basins of attraction of GMACA Fundamental results to characterize GMACA rules Reverse engineering method to synthesize GMACA Combination of reverse engineering method and GA Higher memorizing capacity than Hopfield network Problems: Can handle only binary patterns Solutions: Fuzzy MACA (FMACA)

Fuzzy Cellular Automata (FCA) A linear array of cells Each cell assumes a state - a rational value in [0, 1] Combines both fuzzy logic and Cellular Automata Out of 256 rules, 16 rules are OR and NOR rules (including 0 and 255) Boolean Function Opeartion FCA Operation OR (a + b) min{1, (a + b)} AND (a.b) (a.b) NOT (~a) (1 a)

Fuzzy Cellular Automata (FCA) OR Logic NOR Logic Rule 170 : q I (t+1) = q I-1 (t) Rule 204 : q I (t+1) = q I (t) Rule 85 : q I (t+1) = q I-1 (t) Rule 51 : q I (t+1) = q I (t) Rule 238 : q I (t+1) = q I (t) + q I+1 (t) Rule 17 : q I (t+1) = q I (t) + q I+1 (t) Rule 240 : q I (t+1) = q I+1 (t) Rule 15 : q I (t+1) = q I+1 (t) Rule 250 : q I (t+1) = q I-1 (t) + q I+1 (t) Rule 252 : q I (t+1) = q I-1 (t) + q I (t) Rule 5 : q I (t+1) = q I-1 (t) + q I+1 (t) Rule 3 : q I (t+1) = q I-1 (t) + q I (t) Rule 254 : q I (t+1) = q I-1 (t) + q I (t) + q I+1 (t) Rule 1 : q I (t+1) = q I-1 (t) + q I (t) + q I+1 (t)

Fuzzy Cellular Automata (FCA) 16 OR and NOR rules can be represented by n x n matrix T and an n dimensional binary vector F S i (t) represents the state of i th cell at t th time instant S i (t+1) = F i -min{1, Σ j T ij.s j (t)} where T 1 if next state of ij = ith cell dependents on j th cell 0 otherwise F = Inversion vector, contains 1 where NOR rule is applied 1 1 0 0 0 4-cell null boundary hybrid FCA <238,1,238,3> T = 1 1 1 0 0 0 1 1 F = 1 0 0 0 1 1 1

Fuzzy Multiple Attractor CA (FMACA)

Fuzzy Multiple Attractor CA (FMACA) Dependency Vector (DV) corresponding to matrix Derived Complement Vector (DCV) corresponding to inversion vector Pivot cell (PC) represents an attractor basin uniquely State of Pivot Cell (PC) of attractor of the basin where a state belongs q m = min {1, Σ j DCV j -DV j.s j (t) } Size of attractor basins equal as well as unequal Matrix, inversion vector from DV/DCV

Fuzzy Multiple Attractor CA (FMACA) 0.0 1.0 1.0 1.0 0.0 0.5 0.5 1.0 1.0 0.5 0.0 0.5 0.0 0.5 0.5 0.5 1.0 0.5 1.0 0.0 1.0 1.0 0.5 1.0 0.5 0.5 1.0 0.5 0.5 0.0 1.0 1.0 0.5 0.5 0.5 0.5 0.0 1.0 0.5 1.0 1.0 1.0 1.0 1.0 0.0 1.0 0.0 0.0 1.0 0.5 0.0 1.0 0.5 0.5 0.0 0.5 1.0 0.5 1.0 0.0 0.0 1.0 0.0 0.5 0.0 1.0 0.0 0.0 1.0 0.0 0.0 0.5 1 1 0 T = 0 0 1 0 1 F = 0 DV = 1 DCV = 0 0 0.0 0.0 0.0 0.0 0.5 0.0 0 0 0 0 1 0 0.5 0.0 0.0

Fuzzy Multiple Attractor CA (FMACA) 0.0 1.0 0.0 1.0 0.0 0.5 0.5 1.0 0.0 0.5 0.0 0.5 0.0 0.5 0.5 0.5 1.0 0.5 1.0 0.0 0.0 1.0 0.5 0.0 0.5 0.5 0.0 0.5 0.5 1.0 1.0 1.0 0.5 0.5 0.5 0.5 0.0 1.0 0.5 1.0 1.0 0.0 1.0 1.0 1.0 1.0 0.0 1.0 1.0 0.5 1.0 1.0 0.5 0.5 0.0 0.5 0.0 0.5 1.0 1.0 0.0 1.0 1.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 1 1 0 T = 0 0 1 0 1 F = 1 DV = 1 DCV = 0 0 0.0 0.0 1.0 0.0 0.5 1.0 0 0 0 1 1 1 0.5 0.0 1.0

FMACA based Tree-Structured Classifier FMACA based tree-structured pattern classifier Can handle binary as well as real valued datasets Provides equal and unequal size of attractor basins Combination of GA and DV/DCV FMACA 1 FMACA 2 II IV FMACA 3 FMACA 4 I III IV I II III I II IV

Experimental Setup Randomly generate K number of centroids Around each centroid, generate t number of tuples 50 % patterns are taken for training 50 % patterns are taken for testing A 1 D min A 2 d max A K

Performance Analysis of FMACA Generalization of FMACA tree Dataset Depth Training Accuracy Testing Accuracy Breadth Depth: Number of layers from root to leaf Breadth: Number of intermediate nodes Can generalize dataset irrespective of classes, tuples, attributes (n=5,k=2,t=4000) 1 2 3 4 16.9 81.8 97.7 98.4 15.6 80.9 91.6 92.4 1 21 35 9 Attributes (n) Size (t) No of Classes FMACA C4.5 6 6000 4 8 96.68 89.26 93.10 81.60 Higher classification accuracy compared to C4.5 8 10000 6 10 85.91 85.61 79.60 73.93 10 10000 4 8 83.82 74.01 77.10 66.90

Performance Analysis of FMACA Generation Time (ms) Retrieval Time (ms) Dataset FMACA C4.5 FMACA C4.5 (n=5,k=2,t=2000) 14215 273 3 306 High generation time - but, one time cost Lower retrieval time compared to C4.5 (n=5,k=2,t=20000) 52557 756 80 812 (n=6,k=2,t=2000) 722725 162 4 259 (n=6,k=2,t=20000) 252458 791 35 874 Attributes (n) Size (t) No of Classes FMACA C4.5 6 6000 4 8 1362 1331 5109 6932 Lower memory overhead compared to C4.5 8 8000 6 8 1261 1532 9519 8943 10 10000 6 10 1364 1481 7727 7984

Performance on STATLOG Dataset Classification Accuracy (%) Dataset FMACA MACA DNA 88. 1 87.9 Satimage 81.9 77.5 Shuttle 93.1 94.1 Letter 85.1 84.4 Number of CA cells FMACA MACA 180 180 36 540 9 567 16 1008 No of Nodes of Tree FMACA MACA 122.1 124.3 161.6 199.2 51.8 47.2 661.8 524.6 Memory Overhead Retrieval Time (ms) Comparable accuracy Lesser CA cells Lesser memory overhead Lesser retrieval time Dataset DNA Satimage Shuttle Letter FMACA 51.22 189.62 18.29 261.08 MACA 50.86 222.74 55.17 354.11 FMACA 491 2419 1982 1382 MACA 494 4943 6633 5633

Conclusion Introduction of fuzzy CA in pattern recognition New mathematical tools Dependency matrix, Dependency vector Complement vector, Derived complement vector Reduction of complexity to identify attractors from O(n 3 ) to O(n) Both equal and unequal size of attractor basins Movement of patterns from one to another basin Reduction of search space Elegant evolutionary algorithm combination of DV/DCV and GA FMACA based tree-structured pattern classifier

Future Extensions Applications in pattern clustering, mix-mode learning Theoretical analysis of memorizing capacity of non-linear CA Combination of fuzzy set and fuzzy CA 1-D CA to 2-D CA Development of hybrid systems using CA CA + neural network + fuzzy set CA + fuzzy set + rough set Boolean CA to multi-valued / hierarchical CA Application of CA in Bioinformatics Medical Image Analysis Image Compression Data Mining

Thank You