Computational Complexity and Genetic Algorithms

Save this PDF as:

Size: px
Start display at page:

Transcription

1 Computational Complexity and Genetic Algorithms BART RYLANDER JAMES FOSTER School of Engineering Department of Computer Science University of Portland University of Idaho Portland, Or Moscow, Idaho U.S.A. U.S.A. Abstract: - Recent theory work has suggested that the genetic algorithm (GA) complexity of a problem can be measured by the growth rate of the minimum problem representation []. This paper describes a method for evaluating the computational complexity of a problem specifically for a GA. In particular, we show that the GAcomplexity of a problem is determined by the growth rate of the minimum representation as the size of the problem instance increases. This measurement is then applied to evaluate the GA-complexity of two dissimilar problems. Once the GA-complexities of the problems are known, they are then compared to what is already known about the problems' complexities. These results lead to the definition of a new complexity class called "NPG". Future directions for research are then proposed. Key-Words: - Complexity, Genetic, Algorithm, GA-complexity Introduction Informally, computational complexity is the study of classes of problems based on the rate of growth of space, time, or other fundamental unit of measure as a function of the size of the input [2]. It seeks to determine what problems are computationally tractable, and to classify problems based on their best possible convergence time over all possible problem instances and algorithms. A complexity analysis of a method of automatic programming such as a GA seeks to answer another question as well. Namely, does the method in question, with all of its methodspecific computational baggage, provide an advantage or disadvantage to solving a particular problem over all other methods. This is in contrast to a study that seeks to identify what features of a problem instance cause the method to converge more slowly, or a study that seeks to analyze the expected convergence time of the method itself. To illustrate this difference, consider the wellknown problem of Sort (i.e. given a list of n elements, arrange the list in some predefined manner). If there is only one element that is out of order then a particular algorithm may converge more quickly than if all the elements are out of order, as is the case with the algorithm Insertion Sort. An analysis based on problem instances seeks to learn what features of the particular instance cause the convergence to occur more slowly (such as the number of elements being out of order). An analysis of the algorithm itself may come to the conclusion that Insertion Sort converges O(n 2 ). The computational complexity analysis is concerned with the complexity of the problem over all possible instances, sizes, and algorithms. In this case, the complexity of Sort is the complexity of the fastest possible algorithm for solving Sort, not just the fastest known algorithm. By confining this analysis to the complexity of problems specifically when an evolutionary algorithm is applied, we can compare our findings to what is already known about the complexity of problems. In this way, we may be able to ascertain when best to apply evolutionary algorithms. This paper is a continuation of an ongoing effort to understand the computational complexity of problems specific to evolutionary algorithms. In particular, it is a report that describes how to determine a problem's GA-complexity. Two example applications are described, as well as the definition of a new complexity class. Finally, future directions for research are suggested. GAs are a biologically inspired method for evolving solutions to particular problems [3]. They are typically used when a more direct form of programming cannot be found. Years of successful application to a wide variety of problems have given ample reason to be optimistic that GAs are an efficient method for solving such problems. This evidence is primarily anecdotal however. Little formal theoretic work has been done to compare the efficiency of GAs to all other methods. Most research has concentrated on what features of a

2 problem instance cause the GA to converge more slowly, or work that seeks to describe what the complexity of the GA as an algorithm is. As described above, computational complexity is a property of a problem and not an algorithm. Consequently, it seems valuable to ascertain what the computational complexity of a problem is specifically when a GA is being used. 2 Minimum Chromosome Length GAs have a probabilistic convergence time [4]. By repeating an experiment a number of times it is possible to determine the average convergence time (typically measured as the number of generations to convergence) for a specific GA for a specific problem instance. This average convergence may be mistaken for the complexity of the problem. However, by increasing the size of the instance we will in many cases arrive at a different convergence time. It is this growth rate as the instance size increases that we are looking for. For example, a problem whose convergence is in O(x 2 ) will converge at a rate equivalent to the square of the instance size. In this example, an instance size of 8 will result in an expected convergence at 64 generations. Consequently our goal is to determine the function that best describes the expected convergence time for a problem across all instance sizes. As an example, consider the problem of Maximum 's (i.e. for a given string length n, find the maximum number of 's that can be held in the string). This is possibly the easiest problem for a GA to implement. Simply let each bit position represent a possible. Then, for each chromosome, count the number of 's to determine that chromosome's fitness. For a string of length 5, the maximum fitness for a chromosome is 5. As the problem instance size increases, the length of the chromosome increases. This means that the size of the search space doubles for every increase in instance size because the number of possible solutions is equivalent to the number 2 raised to the length of the chromosome, or 2 l. Observe that regardless of the expected convergence time per instance size, the growth rate of the search space determines the complexity of the problem for the GA. As an example, imagine the GA will converge after examining /0 of the search space. If the search space doubles for every increase in instance size, the GA-complexity of the problem will be (/0)* 2 l. This is clearly an exponential convergence time. The search space growth rate will dominate over every expected convergence rate regardless of how quickly a particular implementation can converge. However, if the minimum chromosome length doesn't have to change every time the instance size increases, this indicates that the problem convergence and consequently the GA-complexity for the problem are less than exponential time. Note that with the problem Maximum 's there is a smaller representation. A representation can be created in which the maximum number of 's in a string can be encoded in binary. This means that instead of 5 bits being required to represent a string of length 5, 3 bits will suffice (e.g. 0). The growth rate of the search space changes as well. Notice, for our new representation, we don't always have to increase the length of the chromosome. A 5 is encoded with 3 bits. Also, a 6 is encoded with 3 bits. Consequently, there is no change in convergence time when going from a string of length 5 to a string of length 6. In fact, our representation doesn't cause the chromosome length to increase until the instance size is increased from 7 to 8 (because 8 requires 4 bits, e.g. 000). This fact alone suggests that the GAcomplexity of Maximum 's is some constant c times lg(x). In order to validate our theoretical analysis we constructed an experiment for verification. To conduct our study we employed a typical GA implementation strategy. We chose an arbitrarily fixed population of 0 chromosomes, single point crossover, and a fixed mutation rate of % per bit. There was 50% elitism thus providing a 50% survival rate for each generation. Selection was implemented randomly based on the chromosomes that survived. This particular GA implementation worked well with the problems in this study and in fact has worked very well for a variety of problems that it has been applied to. Note that the principle of measuring the growth rate of the minimum length representation can be used with any GA implementation, since what is actually being measured is the growth rate of the search space. Therefore it is not necessary to demonstrate the usefulness of this measure with every GA implementation. We then let the GA run until all chromosomes had converged to the optimum solution. This was repeated 5000 times to produce an average convergence time for a given chromosome length. Then the problem instance was increased and the 5000 runs were repeated. This experiment tested the range of string lengths from 4 to 8,42,376. The

3 length of the chromosomes ranged from 3 bits to 24 bits. See Table. formalized. See Definition for the definition of the Minimum Chromosome Length (MCL). String Length Chrom. Length Average Conver k 8m Definition : For a problem P, and D n the set of instances of P of size n, let MCL(P,n) be the least l for which there is an encoding e:s l D n with a domain dependent evaluation function g, where g and e are in FP (the class of functions computable in polynomial time). Table, the average number of generations to converge per instance size of string length. As can be seen, the convergence time is less than exponential. In fact, the convergence varied linearly with the chromosome length. This particular experiment had a convergence of roughly 3* b(x) (3 times the length in binary of x). Since the chromosome itself grew at lg(x), this means that the convergence is in O(lg(x)). It should be noted that this rate of convergence held for a very wide range. Consequently, unless it is possible to derive a smaller representation, we can assert from both experimental as well as theoretic evidence that the GA-complexity of Maximum 's is O(lg(x)). A graphical depiction of the convergence times over the entire range can be seen in Fig String Length Ave Gens String Length E+05 2E+06 Ave Gens Fig. 2, a logarithmically scaled graph of the average convergence times for the Maximum 's problem. A desideratum for a representation is to minimize the number of bits in a chromosome that still uniquely identifies a solution to the problem. In so doing, one minimizes the search space that must be examined to find an answer. This notion can be That is, MCL(P,n) measures the size of the smallest chromosome for which there is a polynomial time computable representation and evaluation function. Notice that for a fixed problem P, MCL(P,n) is a function of the input instance size n, and so is comparable to classical complexity measures. Also, since a smaller representation means a smaller number of solutions to search, there must be a limit as to how small the representation can become. So MCL(P,n) is well-defined for every P and n. The MCL growth rate can be used to bound the worst case complexity of the problem for a GA. If a problem is in NPO, then the MCL growth rate will be no more than linear, since an NP algorithm can search a space which grows no more quickly than exponentially, and linear MCL growth implies exponential search space growth. Conversely, if MCL grows slowly enough, then the search space will be small enough to place the problem in PO. (PO and NPO are the optimization equivalents of P and NP, which are classes of decision problems.[5]) In particular Theorem : for any problem P, if MCL( P, n) k 2! O( n ) for some k, then P PO. It is currently unclear if the converse is true. However, if 2 MCL(P,n) O(n k ) then the problem is most likely in NPG (the class of problems that take more than polynomial time for a GA to solve). Definition 2: A problem P is in the class NPG if MCL(P,n) O(n). At this point it seems warranted to demonstrate the method on a problem that may fit the requirements for our new complexity class, NPG. Maximum Clique (i.e. for a graph G, find the largest complete subgraph H) is a problem that has been shown to be NP-complete [6]. A representation can be developed in which bit can represent node in the desired graph.

4 Consider the graph G with 5 nodes in Figure 3. We will label the nodes 0 through 4. The maximum clique in G is the set of nodes {,2,3}. For an n-node graph, we will use the n-bit representation that designates subgraph H, where the ith bit of the chromosome is (0) if node i is (is not) in H. 00 represents the maximum clique in our example. This representation has been used several times for this problem [7]. Clearly, this is exponential growth. In addition to the table it may be helpful to see the graph in Figure Nodes Ave Gens Nodes Ave Gens Fig. 3, an example instance of Maximum Clique (MC), with solution {,2,3}. This representation is an MCL for a GA implementation of Maximum Clique, since any smaller representation would be unable to represent all 2 n subsets of nodes for an n-node graph, and each subset is a maximal clique for some input instance. In fact, MCL(MC,n)=n is easy to verify. Given this, we see that Maximum Clique's GAcomplexity is exponential. Every time the instance size is increased, we must add another bit to our representation. To verify our prediction we used the same GA implementation that was used for the Maximum 's problem. The experiment was conducted in the same manner. That is, for each instance size the average number of generations to convergence was calculated from a total of 5000 runs. Unfortunately, due to the exponential nature of the problem, it was impossible to view the problem over the same range of instances. Table 2 contains the results. Nodes Ave Gens Table 2, the average number of generations to converge per instance size of graph. Fig. 4, logarithmically scaled graph of the average convergence times for the Maximum Clique problem. Note that this is a logarithmic graph, which is why the convergence time seems relatively linear. In fact, using a traditional graph, the average convergence time for Maximum Clique would exhibit the classical asymptotic bend that is characteristic of an exponential problem. Note also that for Fig. 2 (Maximum 's) the lower line represents the average generations to converge whereas in Fig. 4 (Maximum Clique), the upper line depicts the average generations to converge. In both of our study problems, the GA-complexity is similar to the general complexity of the problem. Maximum 's is trivially a problem in P. Also, since there is no known polynomial time algorithm for Maximum Clique, the exponential time required for the GA to solve it is not unexpected. Despite these facts it is not impossible or even unlikely that some problems in PO could take more than polynomial time for a GA to solve. A problem could be in PO, despite having a rapidly growing MCL and therefore a rapidly growing search space. This would happen if, for any input instance, there was a small (polynomial in the size of the input) portion of the space that is assured to contain the solution, and this region could be identified in polynomial time. However, the only way this could happen would be if any alternative representation, which maps problem instances, requires more than polynomial time. In a sense, the interesting part of the search space must be efficiently hidden (by the representation) in the actual search space in such a way that no efficient algorithm

5 can discover it. Such a function is a classical one-way function, and it is well known that one-way functions exist iff P NP (actually, iff P UP). This is very close to a proof of: MCL( P, n) k Conjecture : If P PO, and 2! O( n ), then P NP. This is also evidence that if a particular MCL grows linearly, then the underlying problem is most likely in NPO\PO (unless P=NP). After all, we explicitly disallow unnecessarily inefficient representations in our definition of MCL. Put another way, even though the encoding e:s l D n minimizes l, the search space may be larger than the solution space. This happens if some redundant encoding is forced by the requirement that encoding and evaluation be polynomial time. In this case, it would clearly be more efficient to avoid algorithms such as GAs that require a representation. If the solution space for a problem P grows polynomially, while MCL(P,n) grows super-polynomially, then the P is clearly in PO, but a GA would be a poor choice of methods to solve P. 3 Conclusions We have introduced a method (MCL) for evaluating the GA-complexity of problems. It was shown that in using this method it is possible to predict the complexity of problems specific to GAs. This was verified in two specific cases experimentally. In addition, we have laid the foundation for applying this method to other problems and for theoretical analysis. Since this method is based on the growth rate of the search space as a function of the size of the input instance, it is similar to classical measures of problem complexity. One of the most important contributions is that we now have the beginning of a theory that will let us evaluate whether GAs are indeed efficient for all or even any problems. There are many possible directions for future research. Our study used a fixed population of 0 chromosomes. If the population were fixed as a percentage of the search space as the search space is allowed to grow, it seems possible that the convergence time may remain fixed. Also, if the population were some other fixed number of chromosomes, it seems likely that there would be no difference in problem complexity. In addition, there are other problems that may now be tackled. What is a GA-hard problem in this context? In traditional complexity, a Hard problem is one in which all other problems in its class can be reduced to (in other words, solving an instance of the Hard problem would also solve the same size instance of all the others in the class, providing a polynomial time transformation existed). And what about problems that are not Hard for which a GA must take exponential time where other methods need only polynomial time? Would these problems be Semihard? Finally, it is unlikely that MCL is computable, consequently each case must be proven inductively. However, it may be possible to arrive at an arbitrarily good approximation of a problem's MCL. But this too must wait till another day. References: [] Rylander, B., Foster, J., GA-hard Problems, Proc. On Genetic and Evolutionary Computation Conference, [2] Bovet, D., Crescenzi, P. Computational Complexity, Prentice Hall.994. [3] Holland, J., Adaptation in Natural and Artificial Systems, Ann Arbor, MI, University of Michigan Press, 975. [4] Ankenbrandt, C., An Extension To the Theory of Convergence and A Proof of the Time Complexity of Genetic Algorithms, Foundations of Genetic Algorithms, Morgan Kaufman 99, pp [5] Balcàzar, J., Díaz, J., and Gabarró, J. Structural Complexity Theory I, Springer-Verlag, 988. [6] Håstad, J. (996). Clique is Indeed Hard,Proc.on STOC, 996. [7]Marconi, J., Foster, J., Finding Cliques in Keller Graphs with Genetic Algorithms, Proc. International Conference on Evolutionary Computing, 998.

Chapter 8: Introduction to Evolutionary Computation

Computational Intelligence: Second Edition Contents Some Theories about Evolution Evolution is an optimization process: the aim is to improve the ability of an organism to survive in dynamically changing

NP Complete Problems. COMP 215 Lecture 20

NP Complete Problems COMP 215 Lecture 20 Complexity Theory Complexity theory is a research area unto itself. The central project is classifying problems as either tractable or intractable. Tractable Worst

INVARIANT SUBSETS OF THE SEARCH SPACE AND THE UNIVERSALITY OF A GENERALIZED GENETIC ALGORITHM

INVARIANT SUBSETS OF THE SEARCH SPACE AND THE UNIVERSALITY OF A GENERALIZED GENETIC ALGORITHM BORIS MITAVSKIY Abstract In this paper we shall give a mathematical description of a general evolutionary heuristic

K-center Hardness and Max-Coverage (Greedy)

IOE 691: Approximation Algorithms Date: 01/11/2017 Lecture Notes: -center Hardness and Max-Coverage (Greedy) Instructor: Viswanath Nagarajan Scribe: Sentao Miao 1 Overview In this lecture, we will talk

Notes for Lecture 2. Statement of the PCP Theorem and Constraint Satisfaction

U.C. Berkeley Handout N2 CS294: PCP and Hardness of Approximation January 23, 2006 Professor Luca Trevisan Scribe: Luca Trevisan Notes for Lecture 2 These notes are based on my survey paper [5]. L.T. Statement

P, NP, NP-Complete, and NPhard

P, NP, NP-Complete, and NPhard Problems Zhenjiang Li 21/09/2011 Outline Algorithm time complicity P and NP problems NP-Complete and NP-Hard problems Algorithm time complicity Outline What is this course

CMPSCI611: NP Completeness Lecture 17 Essential facts about NP-completeness: Any NP-complete problem can be solved by a simple, but exponentially slow algorithm. We don t have polynomial-time solutions

the tree till a class assignment is reached

Decision Trees Decision Tree for Playing Tennis Prediction is done by sending the example down Prediction is done by sending the example down the tree till a class assignment is reached Definitions Internal

Unit 1A: Computational Complexity

Unit 1A: Computational Complexity Course contents: Computational complexity NP-completeness Algorithmic Paradigms Readings Chapters 3, 4, and 5 Unit 1A 1 O: Upper Bounding Function Def: f(n)= O(g(n)) if

Computer Science 385 Analysis of Algorithms Siena College Spring Topic Notes: Limitations of Algorithms

Computer Science 385 Analysis of Algorithms Siena College Spring 2011 Topic Notes: Limitations of Algorithms We conclude with a discussion of the limitations of the power of algorithms. That is, what kinds

Applying Bayesian networks in the game of Minesweeper

Applying Bayesian networks in the game of Minesweeper Marta Vomlelová Faculty of Mathematics and Physics Charles University in Prague http://kti.mff.cuni.cz/~marta/ Jiří Vomlel Institute of Information

Lecture 4 October 18th

Directed and undirected graphical models Fall 2017 Lecture 4 October 18th Lecturer: Guillaume Obozinski Scribe: In this lecture, we will assume that all random variables are discrete, to keep notations

CS 350 Algorithms and Complexity

1 CS 350 Algorithms and Complexity Fall 2015 Lecture 15: Limitations of Algorithmic Power Introduction to complexity theory Andrew P. Black Department of Computer Science Portland State University Lower

A Probabilistic Algorithm for -SAT Based on Limited Local Search and Restart

A Probabilistic Algorithm for -SAT Based on Limited Local Search and Restart Uwe Schöning Universität Ulm, Abteilung Theoretische Informatik James-Franck-Ring, D-89069 Ulm, Germany e-mail: schoenin@informatik.uni-ulm.de

1 The linear algebra of linear programs (March 15 and 22, 2015)

1 The linear algebra of linear programs (March 15 and 22, 2015) Many optimization problems can be formulated as linear programs. The main features of a linear program are the following: Variables are real

The Test Vector Problem and Limitations to Evolving Digital Circuits

The Test Vector Problem and Limitations to Evolving Digital Circuits Kosuke Imamura James A. Foster Axel W. Krings Computer Science Department University of Idaho Moscow, ID 83844-1010 {kosuke,foster,krings}@cs.uidaho.edu

Approximating the step change point of the process fraction nonconforming using genetic algorithm to optimize the likelihood function

Journal of Industrial and Systems Engineering Vol. 7, No., pp 8-28 Autumn 204 Approximating the step change point of the process fraction nonconforming using genetic algorithm to optimize the likelihood

A GA Mechanism for Optimizing the Design of attribute-double-sampling-plan

A GA Mechanism for Optimizing the Design of attribute-double-sampling-plan Tao-ming Cheng *, Yen-liang Chen Department of Construction Engineering, Chaoyang University of Technology, Taiwan, R.O.C. Abstract

DETECTING THE FAULT FROM SPECTROGRAMS BY USING GENETIC ALGORITHM TECHNIQUES

DETECTING THE FAULT FROM SPECTROGRAMS BY USING GENETIC ALGORITHM TECHNIQUES Amin A. E. 1, El-Geheni A. S. 2, and El-Hawary I. A **. El-Beali R. A. 3 1 Mansoura University, Textile Department 2 Prof. Dr.

CS5371 Theory of Computation. Lecture 19: Complexity IV (More on NP, NP-Complete)

CS5371 Theory of Computation Lecture 19: Complexity IV (More on NP, NP-Complete) Objectives More discussion on the class NP Cook-Levin Theorem The Class NP revisited Recall that NP is the class of language

Testing Problems with Sub-Learning Sample Complexity

Testing Problems with Sub-Learning Sample Complexity Michael Kearns AT&T Labs Research 180 Park Avenue Florham Park, NJ, 07932 mkearns@researchattcom Dana Ron Laboratory for Computer Science, MIT 545 Technology

Improved TBL algorithm for learning context-free grammar

Proceedings of the International Multiconference on ISSN 1896-7094 Computer Science and Information Technology, pp. 267 274 2007 PIPS Improved TBL algorithm for learning context-free grammar Marcin Jaworski

CSC 4510 Machine Learning

10: Gene(c Algorithms CSC 4510 Machine Learning Dr. Mary Angela Papalaskari Department of CompuBng Sciences Villanova University Course website: www.csc.villanova.edu/~map/4510/ Slides of this presenta(on

4.1 Notation and probability review

Directed and undirected graphical models Fall 2015 Lecture 4 October 21st Lecturer: Simon Lacoste-Julien Scribe: Jaime Roquero, JieYing Wu 4.1 Notation and probability review 4.1.1 Notations Let us recall

6.867 Machine learning, lecture 23 (Jaakkola)

Lecture topics: Markov Random Fields Probabilistic inference Markov Random Fields We will briefly go over undirected graphical models or Markov Random Fields (MRFs) as they will be needed in the context

Learning Bayesian Networks Does Not Have to Be NP-Hard

Learning Bayesian Networks Does Not Have to Be NP-Hard Norbert Dojer Institute of Informatics, Warsaw University, Banacha, 0-097 Warszawa, Poland dojer@mimuw.edu.pl Abstract. We propose an algorithm for

Data Mining. Preamble: Control Application. Industrial Researcher s Approach. Practitioner s Approach. Example. Example. Goal: Maintain T ~Td

Data Mining Andrew Kusiak 2139 Seamans Center Iowa City, Iowa 52242-1527 Preamble: Control Application Goal: Maintain T ~Td Tel: 319-335 5934 Fax: 319-335 5669 andrew-kusiak@uiowa.edu http://www.icaen.uiowa.edu/~ankusiak

Algorithm Design and Analysis

Algorithm Design and Analysis LECTURE 26 Computational Intractability Polynomial Time Reductions Sofya Raskhodnikova S. Raskhodnikova; based on slides by A. Smith and K. Wayne L26.1 What algorithms are

Introduction to Advanced Results Master Informatique Université Paris 5 René Descartes 2016 Master Info. Complexity Advanced Results 1/26 Outline Boolean Hierarchy Probabilistic Complexity Parameterized

Optimal Fractal Coding is NP-Hard 1

Optimal Fractal Coding is NP-Hard 1 (Extended Abstract) Matthias Ruhl, Hannes Hartenstein Institut für Informatik, Universität Freiburg Am Flughafen 17, 79110 Freiburg, Germany ruhl,hartenst@informatik.uni-freiburg.de

A New Variation of Hat Guessing Games

A New Variation of Hat Guessing Games Tengyu Ma 1, Xiaoming Sun 1, and Huacheng Yu 1 Institute for Theoretical Computer Science Tsinghua University, Beijing, China Abstract. Several variations of hat guessing

Some Remarks on Alternating Temporal Epistemic Logic

Some Remarks on Alternating Temporal Epistemic Logic Corrected version: July 2003 Wojciech Jamroga Parlevink Group, University of Twente, Netherlands Institute of Mathematics, University of Gdansk, Poland

Evolving Presentations of Genetic Information: Motivation, Methods, and Analysis

Evolving Presentations of Genetic Information: Motivation, Methods, and Analysis Peter Lee Stanford University PO Box 14832 Stanford, CA 94309-4832 (650)497-6826 peterwlee@stanford.edu June 5, 2002 Abstract

Feasibility-Preserving Crossover for Maximum k-coverage Problem

Feasibility-Preserving Crossover for Maximum -Coverage Problem Yourim Yoon School of Computer Science & Engineering Seoul National University Sillim-dong, Gwana-gu Seoul, 151-744, Korea yryoon@soar.snu.ac.r

DRAFT. Diagonalization. Chapter 4

Chapter 4 Diagonalization..the relativized P =?NP question has a positive answer for some oracles and a negative answer for other oracles. We feel that this is further evidence of the difficulty of the

Dynamic Call Center Routing Policies Using Call Waiting and Agent Idle Times Online Supplement

Submitted to imanufacturing & Service Operations Management manuscript MSOM-11-370.R3 Dynamic Call Center Routing Policies Using Call Waiting and Agent Idle Times Online Supplement (Authors names blinded

The Complexity of Maximum. Matroid-Greedoid Intersection and. Weighted Greedoid Maximization

Department of Computer Science Series of Publications C Report C-2004-2 The Complexity of Maximum Matroid-Greedoid Intersection and Weighted Greedoid Maximization Taneli Mielikäinen Esko Ukkonen University

The Test Vector Problem and Limitations to Evolving Digital Circuits

Abstract The Test Vector Problem and Limitations to Evolving Digital Circuits Kosuke Imamura James A. Foster Computer Science Computer Science U. Idaho U. Idaho Moscow, ID Moscow, ID 83844-1010 83844-1010

Evolutionary Computation. DEIS-Cesena Alma Mater Studiorum Università di Bologna Cesena (Italia)

Evolutionary Computation DEIS-Cesena Alma Mater Studiorum Università di Bologna Cesena (Italia) andrea.roli@unibo.it Evolutionary Computation Inspiring principle: theory of natural selection Species face

Simulated Annealing for Constrained Global Optimization

Monte Carlo Methods for Computation and Optimization Final Presentation Simulated Annealing for Constrained Global Optimization H. Edwin Romeijn & Robert L.Smith (1994) Presented by Ariel Schwartz Objective

Distance Metrics and Fitness Distance Analysis for the Capacitated Vehicle Routing Problem

MIC2005. The 6th Metaheuristics International Conference 603 Metrics and Analysis for the Capacitated Vehicle Routing Problem Marek Kubiak Institute of Computing Science, Poznan University of Technology

ON THE NP-COMPLETENESS OF SOME GRAPH CLUSTER MEASURES

ON THE NP-COMPLETENESS OF SOME GRAPH CLUSTER MEASURES JIŘÍ ŠÍMA AND SATU ELISA SCHAEFFER Academy of Sciences of the Czech Republic Helsinki University of Technology, Finland elisa.schaeffer@tkk.fi SOFSEM

Hardness of Approximation

Hardness of Approximation We have seen several methods to find approximation algorithms for NP-hard problems We have also seen a couple of examples where we could show lower bounds on the achievable approxmation

Problem. Problem Given a dictionary and a word. Which page (if any) contains the given word? 3 / 26

Binary Search Introduction Problem Problem Given a dictionary and a word. Which page (if any) contains the given word? 3 / 26 Strategy 1: Random Search Randomly select a page until the page containing

Abstract. Three Methods and Their Limitations. N-1 Experiments Suffice to Determine the Causal Relations Among N Variables

N-1 Experiments Suffice to Determine the Causal Relations Among N Variables Frederick Eberhardt Clark Glymour 1 Richard Scheines Carnegie Mellon University Abstract By combining experimental interventions

Classical Complexity and Fixed-Parameter Tractability of Simultaneous Consecutive Ones Submatrix & Editing Problems

Classical Complexity and Fixed-Parameter Tractability of Simultaneous Consecutive Ones Submatrix & Editing Problems Rani M. R, Mohith Jagalmohanan, R. Subashini Binary matrices having simultaneous consecutive

Lecture 15: Genetic Algorithms

Lecture 15: Genetic Algorithms Dr Roman V Belavkin BIS3226 Contents 1 Combinatorial Problems 1 2 Natural Selection 2 3 Genetic Algorithms 3 31 Individuals and Population 3 32 Fitness Functions 3 33 Encoding

Multi-coloring and Mycielski s construction

Multi-coloring and Mycielski s construction Tim Meagher Fall 2010 Abstract We consider a number of related results taken from two papers one by W. Lin [1], and the other D. C. Fisher[2]. These articles

Not all counting problems are efficiently approximable. We open with a simple example.

Chapter 7 Inapproximability Not all counting problems are efficiently approximable. We open with a simple example. Fact 7.1. Unless RP = NP there can be no FPRAS for the number of Hamilton cycles in a

Sorting Network Development Using Cellular Automata

Sorting Network Development Using Cellular Automata Michal Bidlo, Zdenek Vasicek, and Karel Slany Brno University of Technology, Faculty of Information Technology Božetěchova 2, 61266 Brno, Czech republic

Computational Aspects of Aggregation in Biological Systems

Computational Aspects of Aggregation in Biological Systems Vladik Kreinovich and Max Shpak University of Texas at El Paso, El Paso, TX 79968, USA vladik@utep.edu, mshpak@utep.edu Summary. Many biologically

Locally Expanding Hypergraphs and the Unique Games Conjecture

Locally Expanding Hypergraphs and the Unique Games Conjecture Eric Purdy September 25, 2008 Abstract We examine the hardness of approximating constraint satisfaction problems with k-variable constraints,

Bounding the Population Size in XCS to Ensure Reproductive Opportunities

Bounding the Population Size in XCS to Ensure Reproductive Opportunities Martin V. Butz and David E. Goldberg Illinois Genetic Algorithms Laboratory (IlliGAL) University of Illinois at Urbana-Champaign

Lecture 18: More NP-Complete Problems

6.045 Lecture 18: More NP-Complete Problems 1 The Clique Problem a d f c b e g Given a graph G and positive k, does G contain a complete subgraph on k nodes? CLIQUE = { (G,k) G is an undirected graph with

Protein Structure Prediction II Lecturer: Serafim Batzoglou Scribe: Samy Hamdouche

Protein Structure Prediction II Lecturer: Serafim Batzoglou Scribe: Samy Hamdouche The molecular structure of a protein can be broken down hierarchically. The primary structure of a protein is simply its

Computer Sciences Department

Computer Sciences Department 1 Reference Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER Computer Sciences Department 3 ADVANCED TOPICS IN C O M P U T A B I L I T Y

Complexity and NP-completeness

Lecture 17 Complexity and NP-completeness Supplemental reading in CLRS: Chapter 34 As an engineer or computer scientist, it is important not only to be able to solve problems, but also to know which problems

The space complexity of a standard Turing machine. The space complexity of a nondeterministic Turing machine

298 8. Space Complexity The space complexity of a standard Turing machine M = (Q,,,, q 0, accept, reject) on input w is space M (w) = max{ uav : q 0 w M u q av, q Q, u, a, v * } The space complexity of

Crossover Gene Selection by Spatial Location

Crossover Gene Selection by Spatial Location ABSTRACT Dr. David M. Cherba Computer Science Department Michigan State University 3105 Engineering Building East Lansing, MI 48823 USA cherbada@cse.msu.edu

Bounded Approximation Algorithms

Bounded Approximation Algorithms Sometimes we can handle NP problems with polynomial time algorithms which are guaranteed to return a solution within some specific bound of the optimal solution within

Operations and Supply Chain Management Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras

Operations and Supply Chain Management Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras Lecture - 27 Flow Shop Scheduling - Heuristics - Palmer, Campbell Dudek

Ma/CS 117c Handout # 5 P vs. NP

Ma/CS 117c Handout # 5 P vs. NP We consider the possible relationships among the classes P, NP, and co-np. First we consider properties of the class of NP-complete problems, as opposed to those which are

DRAFT. Algebraic computation models. Chapter 14

Chapter 14 Algebraic computation models Somewhat rough We think of numerical algorithms root-finding, gaussian elimination etc. as operating over R or C, even though the underlying representation of the

Time Complexity (1) CSCI Spring Original Slides were written by Dr. Frederick W Maier. CSCI 2670 Time Complexity (1)

Time Complexity (1) CSCI 2670 Original Slides were written by Dr. Frederick W Maier Spring 2014 Time Complexity So far we ve dealt with determining whether or not a problem is decidable. But even if it

CSC 5170: Theory of Computational Complexity Lecture 5 The Chinese University of Hong Kong 8 February 2010

CSC 5170: Theory of Computational Complexity Lecture 5 The Chinese University of Hong Kong 8 February 2010 So far our notion of realistic computation has been completely deterministic: The Turing Machine

Fundamental Dominations in Graphs

Fundamental Dominations in Graphs arxiv:0808.4022v1 [math.co] 29 Aug 2008 Arash Behzad University of California, LosAngeles abehzad@ee.ucla.edu Mehdi Behzad Shahid Beheshti University, Iran mbehzad@sharif.edu

A Duality between Clause Width and Clause Density for SAT

A Duality between Clause Width and Clause Density for SAT Chris Calabro, Russell Impagliazzo, Ramamohan Paturi University of California, San Diego 9500 Gilman Drive, La Jolla, CA 92093 ccalabro,russell,paturi

30 Classification of States

30 Classification of States In a Markov chain, each state can be placed in one of the three classifications. 1 Since each state falls into one and only one category, these categories partition the states.

Bounds on the OBDD-Size of Integer Multiplication via Universal Hashing

Bounds on the OBDD-Size of Integer Multiplication via Universal Hashing Philipp Woelfel Dept. of Computer Science University Dortmund D-44221 Dortmund Germany phone: +49 231 755-2120 fax: +49 231 755-2047

BDD Based Upon Shannon Expansion

Boolean Function Manipulation OBDD and more BDD Based Upon Shannon Expansion Notations f(x, x 2,, x n ) - n-input function, x i = or f xi=b (x,, x n ) = f(x,,x i-,b,x i+,,x n ), b= or Shannon Expansion

VIII. NP-completeness

VIII. NP-completeness 1 / 15 NP-Completeness Overview 1. Introduction 2. P and NP 3. NP-complete (NPC): formal definition 4. How to prove a problem is NPC 5. How to solve a NPC problem: approximate algorithms

Quantum Annealing and the Satisfiability Problem

arxiv:1612.7258v1 [quant-ph] 21 Dec 216 Quantum Annealing and the Satisfiability Problem 1. Introduction Kristen L PUDENZ 1, Gregory S TALLANT, Todd R BELOTE, and Steven H ADACHI Lockheed Martin, United

1 Computational problems

80240233: Computational Complexity Lecture 1 ITCS, Tsinghua Univesity, Fall 2007 9 October 2007 Instructor: Andrej Bogdanov Notes by: Andrej Bogdanov The aim of computational complexity theory is to study

Consistency Under Sampling of Exponential Random Graph Models

Consistency Under Sampling of Exponential Random Graph Models Cosma Shalizi and Alessandro Rinaldo Summary by: Elly Kaizar Remember ERGMs (Exponential Random Graph Models) Exponential family models Sufficient

When to Use Bit-Wise Neutrality

When to Use Bit-Wise Neutrality Tobias Friedrich Department 1: Algorithms and Complexity Max-Planck-Institut für Informatik Saarbrücken, Germany Frank Neumann Department 1: Algorithms and Complexity Max-Planck-Institut

Average Case Analysis of QuickSort and Insertion Tree Height using Incompressibility

Average Case Analysis of QuickSort and Insertion Tree Height using Incompressibility Tao Jiang, Ming Li, Brendan Lucier September 26, 2005 Abstract In this paper we study the Kolmogorov Complexity of a

Computational Intelligence in Product-line Optimization

Computational Intelligence in Product-line Optimization Simulations and Applications Peter Kurz peter.kurz@tns-global.com June 2017 Restricted use Restricted use Computational Intelligence in Product-line

Notes for Lecture 3... x 4

Stanford University CS254: Computational Complexity Notes 3 Luca Trevisan January 18, 2012 Notes for Lecture 3 In this lecture we introduce the computational model of boolean circuits and prove that polynomial

Feature selection and classifier performance in computer-aided diagnosis: The effect of finite sample size

Feature selection and classifier performance in computer-aided diagnosis: The effect of finite sample size Berkman Sahiner, a) Heang-Ping Chan, Nicholas Petrick, Robert F. Wagner, b) and Lubomir Hadjiiski

Boolean circuits. Lecture Definitions

Lecture 20 Boolean circuits In this lecture we will discuss the Boolean circuit model of computation and its connection to the Turing machine model. Although the Boolean circuit model is fundamentally

CS 6820 Fall 2014 Lectures, October 3-20, 2014

Analysis of Algorithms Linear Programming Notes CS 6820 Fall 2014 Lectures, October 3-20, 2014 1 Linear programming The linear programming (LP) problem is the following optimization problem. We are given

QUANTUM COMMUNICATIONS BASED ON QUANTUM HASHING. Alexander Vasiliev. Kazan Federal University

QUANTUM COMMUNICATIONS BASED ON QUANTUM HASHING Alexander Vasiliev Kazan Federal University Abstract: In this paper we consider an application of the recently proposed quantum hashing technique for computing

Forecasting & Futurism

Article from: Forecasting & Futurism December 2013 Issue 8 A NEAT Approach to Neural Network Structure By Jeff Heaton Jeff Heaton Neural networks are a mainstay of artificial intelligence. These machine-learning

The Polynomial Hierarchy

The Polynomial Hierarchy Slides based on S.Aurora, B.Barak. Complexity Theory: A Modern Approach. Ahto Buldas Ahto.Buldas@ut.ee Motivation..synthesizing circuits is exceedingly difficulty. It is even

A.Antonopoulos 18/1/2010

Class DP Basic orems 18/1/2010 Class DP Basic orems 1 Class DP 2 Basic orems Class DP Basic orems TSP Versions 1 TSP (D) 2 EXACT TSP 3 TSP COST 4 TSP (1) P (2) P (3) P (4) DP Class Class DP Basic orems

Efficiency of genetic algorithm and determination of ground state energy of impurity in a spherical quantum dot

Efficiency of genetic algorithm and determination of ground state energy of impurity in a spherical quantum dot +DOXNùDIDN 1* 0HKPHWùDKLQ 1, Berna Gülveren 1, Mehmet Tomak 1 Selcuk University, Faculty

W -Hardness under Linear FPT-Reductions: Structural Properties and Further Applications

W -Hardness under Linear FPT-Reductions: Structural Properties and Further Applications Jianer Chen 1 Xiuzhen Huang 2 Iyad A. Kanj 3 Ge Xia 4 1 Dept. of Computer Science, Texas A&M University, College

Artificial Intelligence (AI) Common AI Methods. Training. Signals to Perceptrons. Artificial Neural Networks (ANN) Artificial Intelligence

Artificial Intelligence (AI) Artificial Intelligence AI is an attempt to reproduce intelligent reasoning using machines * * H. M. Cartwright, Applications of Artificial Intelligence in Chemistry, 1993,

Experimental Supplements to the Theoretical Analysis of EAs on Problems from Combinatorial Optimization

Experimental Supplements to the Theoretical Analysis of EAs on Problems from Combinatorial Optimization Patrick Briest, Dimo Brockhoff, Bastian Degener, Matthias Englert, Christian Gunia, Oliver Heering,

Lecture 29: Tractable and Intractable Problems

Lecture 29: Tractable and Intractable Problems Aims: To look at the ideas of polynomial and exponential functions and algorithms; and tractable and intractable problems. Page 1 of 10 To look at ways of

REIHE COMPUTATIONAL INTELLIGENCE COLLABORATIVE RESEARCH CENTER 531

U N I V E R S I T Y OF D O R T M U N D REIHE COMPUTATIONAL INTELLIGENCE COLLABORATIVE RESEARCH CENTER 531 Design and Management of Complex Technical Processes and Systems by means of Computational Intelligence

Genetic Algorithm for Solving the Economic Load Dispatch

International Journal of Electronic and Electrical Engineering. ISSN 0974-2174, Volume 7, Number 5 (2014), pp. 523-528 International Research Publication House http://www.irphouse.com Genetic Algorithm

Schema Theory. David White. Wesleyan University. November 30, 2009

November 30, 2009 Building Block Hypothesis Recall Royal Roads problem from September 21 class Definition Building blocks are short groups of alleles that tend to endow chromosomes with higher fitness

Dominating Set. Chapter 26

Chapter 26 Dominating Set In this chapter we present another randomized algorithm that demonstrates the power of randomization to break symmetries. We study the problem of finding a small dominating set

Dominating Set. Chapter Sequential Greedy Algorithm 294 CHAPTER 26. DOMINATING SET

294 CHAPTER 26. DOMINATING SET 26.1 Sequential Greedy Algorithm Chapter 26 Dominating Set Intuitively, to end up with a small dominating set S, nodes in S need to cover as many neighbors as possible. It

Multiclass Classification-1

CS 446 Machine Learning Fall 2016 Oct 27, 2016 Multiclass Classification Professor: Dan Roth Scribe: C. Cheng Overview Binary to multiclass Multiclass SVM Constraint classification 1 Introduction Multiclass

Big O 2/14/13. Administrative. Does it terminate? David Kauchak cs302 Spring 2013

/4/3 Administrative Big O David Kauchak cs3 Spring 3 l Assignment : how d it go? l Assignment : out soon l CLRS code? l Videos Insertion-sort Insertion-sort Does it terminate? /4/3 Insertion-sort Loop

CISC 876: Kolmogorov Complexity

March 27, 2007 Outline 1 Introduction 2 Definition Incompressibility and Randomness 3 Prefix Complexity Resource-Bounded K-Complexity 4 Incompressibility Method Gödel s Incompleteness Theorem 5 Outline