Efficient Techniques for Fast Packet Classification
|
|
- Rose Griffin
- 5 years ago
- Views:
Transcription
1 Efficient Techniques for Fast Packet Classification Network Reading Group Alok Tongaonkar, R Sekar Stony Brook University Sept 16, 2008
2 What is Packet Classification? Packet Classification A mechanism that inspects network packets determines how to process a packet based on the values of header fields and/or the payload. Fundamental Operation Identify the rules R i that match a packet p from rules {R 1,..., R n } where R i : condition action e.g., R 1 : dhost = PLUTO && dport = HTTP && content: Bad command DENY
3 Applications Firewalls Identify highest priority matching rule Intrusion Detection Systems Use unordered rules Identify all matching rules Network Monitoring Packet-filtering whether a packet satisfies any of the conditions
4 Previous Techniques Naive technique: Berkeley Packet Filter(BPF) Match one rule at a time A test that occurs in multiple rules is tested once on behalf of each of the rules
5 Previous Techniques Trie-based techniques: PathFinder, Dynamic Packet Filter(DPF) Identify common prefixes and share them {F1, F2} type = IP {F1, F2} proto = T CP {F1, F2} dport = A dport = B {F1} {F2}
6 Previous Techniques DAG automaton: Berkeley Packet Filter(BPF+) Recognize some equivalent states Use data flow analysis to eliminate tests that are implied by other tests performed previously on the path Shost X? Shost X? T T F Dhost Y? F Dhost Y? Shost Y? F Shost Y? T T T T F Dhost X? F Dhost X? F T F T {F1} {F2} {F1} {F2}
7 Previous Techniques Adaptive Traversal Change order of testing to promote sharing {p1, p2, p3} {p1, p2, p3} x = a x a {p1, p2, p3} {p1, p3} y = b y = a y a && y b y = b y = a y a && y b y = b y a && y b y = a {p1} x = a {p2, p3} x a φ {p1} {p2, p3} φ {p1} {p3} φ {p2, p3} {p3}
8 Objective Promote Sharing of Tests Adaptive automata traversal developed in the context of term-matching Restricted to equality tests we need to support inequalities, disequalities, and bit-masking operations Several new techniques in the context of the application domain Flexibility to Support Diverse Applications Ordered (firewalls) and unordered (intrusion detection) rulesets Packet-filtering (network monitoring)
9 Organization of Talk
10 Organization of Talk Part I - Packet Field Matching Algorithm Techniques Intrusion Detection Systems Firewalls Evaluation
11 Organization of Talk Part I - Packet Field Matching Algorithm Techniques Intrusion Detection Systems Firewalls Evaluation Part II - Content Matching Integrating String Matching
12 Organization of Talk Part I - Packet Field Matching Algorithm Techniques Intrusion Detection Systems Firewalls Evaluation Part II - Content Matching Part III Integrating String Matching Related Work Summary
13 Techniques for Packet Classification Naive technique A test that occurs in multiple rules is tested once on behalf of each of the rules Automata-based techniques Automaton states used to remember tests Avoids repetition of tests
14 Deterministic Packet Classification Automaton F 1 : (icmp type = ECHO) F 2 : (icmp type = ECHO REPLY ) (ttl = 1) F 3 : (ttl = 1) icmp type = ECHO {F1, F3} {F1, F3} ttl = 1 ttl 1 ttl 1 {F1} {F2, F3} {F1, F2, F3} icmp type = ECHO REP LY ttl = 1 icmp type ECHO REP LY icmp type ECHO φ ttl 1 {F3} ttl = 1 {F3} All but one transitions labeled with test Remaining transition labeled other conjunction of negations of all tests on the rest of the transitions {F2, F3} φ
15 Deterministic Packet Classification Automaton F 1 : (icmp type = ECHO) F 2 : (icmp type = ECHO REPLY ) (ttl = 1) F 3 : (ttl = 1) icmp type = ECHO {F1, F3} {F1, F3} ttl = 1 ttl 1 ttl 1 {F1} {F2, F3} {F2, F3} {F1, F2, F3} icmp type = ECHO REP LY ttl = 1 icmp type ECHO REP LY icmp type ECHO φ ttl 1 φ {F3} ttl = 1 {F3} Transitions are simultaneously distinguishable All tests except other are mutually exclusive Applicable transition can be determined using a single operation O(1) expected time complexity
16 Deterministic Packet Classification Automaton F 1 : (icmp type = ECHO) F 2 : (icmp type = ECHO REPLY ) (ttl = 1) F 3 : (ttl = 1) icmp type = ECHO {F1, F3} ttl = 1 ttl 1 ttl 1 {F2, F3} {F1, F2, F3} icmp type = ECHO REP LY icmp type ECHO REP LY icmp type ECHO {F3} ttl = 1 Each final state S correctly identifies the match set corresponding to any packet satisfying all the tests along a path from the start state to S. {F1, F3} {F1} ttl = 1 φ ttl 1 {F3} {F2, F3} φ
17 Non-deterministic Packet Classification Automaton F 1 : (icmp type = ECHO) F 2 : (icmp type = ECHO REPLY ) (ttl = 1) F 3 : (ttl = 1) {F1, F2, F3} icmp type ECHO REP LY icmp type = ECHO icmp type = ECHO REP LY {F1} {F3} ttl = 1 ttl 1 {F2, F3} φ {F3} ttl = 1 ttl 1 other conjunction of negations of a subset of tests on the rest of the transitions Nondeterminism is simulated using backtracking at runtime {F2, F3} φ
18 Principal Design Criteria for PCA Operate in real-time on high-speed networks without dropping packets Scale to support thousands of rules typical in intrusion detection systems and firewalls Computational Issues Matching time closely related to path lengths Memory size of automata
19 Problem Formulation Tests Involve a variable x and one or two constants (denoted by c).
20 Problem Formulation Tests Involve a variable x and one or two constants (denoted by c). Equality tests x = c tcp sport = 80
21 Problem Formulation Tests Involve a variable x and one or two constants (denoted by c). Equality tests x = c tcp sport = 80 Equality tests with bitmasks x&c 1 = c tcp flags & 0x03 = 0x03
22 Problem Formulation Tests Involve a variable x and one or two constants (denoted by c). Equality tests x = c tcp sport = 80 Equality tests with bitmasks x&c 1 = c tcp flags & 0x03 = 0x03 Disequality tests x c tcp sport 80
23 Problem Formulation Tests Involve a variable x and one or two constants (denoted by c). Equality tests x = c tcp sport = 80 Equality tests with bitmasks x&c 1 = c tcp flags & 0x03 = 0x03 Disequality tests x c tcp sport 80 Disequality tests with bitmasks x&c 1 c tcp flags & 0x03 0x03
24 Problem Formulation Tests Involve a variable x and one or two constants (denoted by c). Equality tests x = c tcp sport = 80 Equality tests with bitmasks x&c 1 = c tcp flags & 0x03 = 0x03 Disequality tests x c tcp sport 80 Disequality tests with bitmasks x&c 1 c tcp flags & 0x03 0x03 Inequality tests x c or x c tcp dport 1024
25 Filters and Priorities A filter F is a conjunction of tests. (dport = 22) (sport 1024) (flags&0xb = 0x3) A set F of filters may be partially ordered by a priority relation. The priority of F is denoted as Pri(F).
26 Filters and Priorities A filter F is a conjunction of tests. (dport = 22) (sport 1024) (flags&0xb = 0x3) A set F of filters may be partially ordered by a priority relation. The priority of F is denoted as Pri(F). A filter F matches a packet p, if: the packet satisfies F, i.e., F(p) is true the packet does not satisfy any rule that has higher priority than F
27 Filters and Priorities A filter F is a conjunction of tests. (dport = 22) (sport 1024) (flags&0xb = 0x3) A set F of filters may be partially ordered by a priority relation. The priority of F is denoted as Pri(F). Match Set of p consists of all filters that match p, with the exception that among equal priority filters, at most one is retained.
28 Example of Prioritized Matching F 1 : (icmp type = ECHO) F 2 : (icmp type = ECHO REPLY ) (ttl = 1) F 3 : (ttl = 1) p 1 : icmp echo packet with ttl of 1 p 2 : icmp reply packet with ttl of 1
29 Example of Prioritized Matching F 1 : (icmp type = ECHO) F 2 : (icmp type = ECHO REPLY ) (ttl = 1) F 3 : (ttl = 1) p 1 : icmp echo packet with ttl of 1 p 2 : icmp reply packet with ttl of 1 Multi-matching (intrusion detection systems) set incomparable priorities M(p 1 ) = {F 1, F 3 } M(p 2 ) = {F 2, F 3 }
30 Example of Prioritized Matching F 1 : (icmp type = ECHO) F 2 : (icmp type = ECHO REPLY ) (ttl = 1) F 3 : (ttl = 1) p 1 : icmp echo packet with ttl of 1 p 2 : icmp reply packet with ttl of 1 Ordered matching (firewalls) assign monotonically decreasing priorities Pri(F 1 ) > Pri(F 2 ) > Pri(F 3 ) M(p 1 ) = {F 1 } M(p 2 ) = {F 2 }
31 Example of Prioritized Matching F 1 : (icmp type = ECHO) F 2 : (icmp type = ECHO REPLY ) (ttl = 1) F 3 : (ttl = 1) p 1 : icmp echo packet with ttl of 1 p 2 : icmp reply packet with ttl of 1 Packet-filtering (network monitoring) set equal priorities Pri(F 1 ) = Pri(F 3 ) = Pri(F 2 ) p 1 can match either F 1 or F 3 p 2 can match either F 2 or F 3
32 Matching Automata Construction Key New Idea Decompose and reorder tests to increase sharing of tests among rules Example F 1 : (x = 5), F 2 : (x & 0x03 1)
33 Matching Automata Construction Key New Idea Decompose and reorder tests to increase sharing of tests among rules Example F 1 : (x = 5), F 2 : (x & 0x03 1) {F1, F2} x = 5 x 5 {F1, F2} x & 0x03 = 1 x & 0x03 1 x & 0x03 = 1 {F2} x & 0x03 1 {F1} {F1, F2} φ {F2}
34 Matching Automata Construction Key New Idea Decompose and reorder tests to increase sharing of tests among rules Example F 1 : (x = 5), F 2 : (x & 0x03 1) {F1, F2} x&0x03 = 1 x & 0x03 1 {F2} x & 0xfc = 4 x & 0xfc 4 {F2} φ
35 Condition Factorization Decomposing filters into combination of more primitive tests Similar to factorization of integers Based on the residue operation analogous to integer division Residue We want to determine if there is a match for a filter C 1 We have so far tested a condition C 2 A residue captures the additional tests that need to be performed at this point to verify C 1
36 Residue Operation Definition (Residue) The residue C 1 /C 2 is another condition C 3 such that: 1 C 2 C 3 C 1 2 C 1 C 2 C 3 Examples C 1 : x [1, 20], C 2 : x [15, 25] C 1 : x [1, 20], C 2 : x = 15 C 1 : x [1, 20], C 2 : x = 35 C 1 : x [1, 20], C 2 : y = 15 C 3 : x 20 C 3 : true C 3 : false C 3 : x [1, 20]
37 Residue Operation Definition (Residue) The residue C 1 /C 2 is another condition C 3 such that: 1 C 2 C 3 C 1 2 C 1 C 2 C 3 Ideally C 3 would be the weakest condition such that (1) holds
38 Residue Operation Definition (Residue) The residue C 1 /C 2 is another condition C 3 such that: 1 C 2 C 3 C 1 2 C 1 C 2 C 3 In Practice We might not want minimal condition since Expensive to compute Inefficient to use contains many disjunctions
39 Residue Operation Definition (Residue) The residue C 1 /C 2 is another condition C 3 such that: 1 C 2 C 3 C 1 2 C 1 C 2 C 3 Example of Approximation C 1 : x [1, 20], C 2 : x 15 C 3 : x [1, 14] x [16, 20] C 3 : x [1, 20]
40 Residue Operation Definition (Residue) The residue C 1 /C 2 is another condition C 3 such that: 1 C 2 C 3 C 1 2 C 1 C 2 C 3 Need for (2) C 3 shouldn t be too strong, or else we may miss matches for C 1 C 1 : x [1, 20], C 2 : x [10, 30] C 3 : x [10, 15] C 3 satisfies (1) but not (2) Will miss match for x [1, 9] or x [16, 20]
41 Computing Residue on Tests T1 T2 T1/T2 Conditions T T true T T false T x = c T [x c] x = c x & c1 = c2 x & c1 = c & c1 c & c1 = c2 false c & c1 c2 x = c x & c1 c2 false c & c1 = c2 x = c x [c1, c2] false c [c1, c2] x c x & c1 = c2 x & c1 c & c1 c & c1 = c2 true c & c1 c2 x c x & c1 c2 true c & c1 = c2 x c x [c1, c2] true (c < c1) (c > c2) x [c1, c2] x [c3, c4] true c1 c3 c4 c2 x [, c2] c1 c3 c2 c4 x [c1, ] c3 c1 c4 c2 x [c1, c2] c3 c1 c2 c4 false (c2 < c3) (c4 < c1) x [c1, c2] x & c3 = c4 false c4 > c2 x & c1 = c2 x & c3 = c4 x & (c1 & c3) = (c2 & c3) c2 & c3 = c1 & c4 false otherwise x & c1 = c2 x [c3, c4] false c2 > c4 x & c1 c2 x & c3 = c4 x & (c1 & c3) (c2 & c3) c2 & c3 = c1 & c4 true otherwise x & c1 c2 x [c3, c4] true c2 > c4 T T T
42 Build Algorithm Recursive procedure Takes an automaton state s as its first parameter Builds the subautomaton that is rooted at s It takes two other parameters C s, the candidate set of the state s M s, the match set of s Candidate Set C s filters that haven t completed a match, but future matches can t be ruled out either. Match Set M s all filters for which a match can be announced at s.
43 Build Algorithm 1. procedure Build(s, C s, M s ) 2. if C s is empty 3. then match[s] = M s 4. else 5. (D, T ) = select(c s ) 6. T o = { d i D d i =true T i} 7. for each T i (T {T o }) do 8. C i = C s /T i 9. if ((T i T o ) d i ) then C i = C i C/T o endif 10. compute M si and C si from C i and M s 11. if a state s i corresponding to (C si, M si ) isn t present 12. create a new state s i 13. Build(s i, C si, M si ) 14. endif 15. create a transition from s to s i on T i 16. end 17. endif
44 Improving Automata Size Key Idea Pick tests which avoid duplication of filters in next states T = {x = 5, x = 6, (x 5) (x 6)} C = {x = 5, x = 6, x > 7} C = {x = 6, x > 4} {C1, C2, C3} X = 5 X = 6 X 5 X 6 X = 5 X = 6 X 5 X 6 {C1} {C2} {C3} {C2} {C1, C2} {C2}
45 Improving Automata Size Key Idea Pick tests which avoid duplication of filters in next states T = {x = 5, x = 6, (x 5) (x 6)} C = {x = 5, x = 6, x > 7} C = {x = 6, x > 4} Definition (Discriminating Set) A set T of conditions is said to be a discriminating set for a filter set F iff for every F F there exists at most one T T such that F belongs to the candidate set of F/T. Concept of discriminating tests is similar to the concept of index in the context of term matching.
46 Ensuring Polynomial-Size Automata Breadth of subautomaton rooted at s k B( C s ) = B( C si ), i=1 P(n) the desired polynomial on n that bounds the automaton size. k P( C s ) P( C si ) (1) i=1 Pick tests that satisfy the bounds Pick a test that comes closest to satisfying this constraint and make some outgoing transitions nondeterministic
47 Benign Nondeterminism Two filters F 1 and F 2 are said to be independent of each if they do not have a common test Build separate automaton for each independent set Match packets against each automaton non-determinism without incurring any performance penalties
48 Effect of Benign Nondeterminism on Automata Size Leads to dramatic reduction in automata size especially for intrusion detection systems. If F 1 and F 2 are independent, packet may match F 1, F 2, both, or neither. Number of states of automaton for F 1 is k 1, for F 2 is k 2. Number of states of automaton for F 1 F 2 is k 1 k 2. Combined number of states of independent automata for F 1 and F 2 is k 1 + k 2.
49 Improving Matching Time Utility How much a test goes towards checking a filter Based on notion of assigning costs to tests and filters Compare cost of a filter with combined cost of a test and the residue of a filter w.r.t the test select strategy Size reduction more important than matching time 1 Pick discriminating test when available Pick test with higher utility 2 Examine opportunities for benign-nondeterminism 3 Pick tests that satisfy polynomial bound
50 Measuring Matching Time Implementation-independent metric for matching time Suppose we could guess the set of rules that match a packet The match verification cost is lower bound for any algorithm that tries to identify the matching rules We use the ratio of actual matching cost to the lower bound for match verification as a metric for matching time
51 Experiments Setup for IDS Snort open source Comprehensive default signatures Signatures consist of packet field tests and content-matching operation Snort Next Generation (Snort-NG) matches packet fields in parallel Snort version 2 (Snort v2) tries to parallelize matching for some fields Used 1635 default rules that come with Snort combined rules with same packet field tests to get 305 rules System: 1.70Ghz pentium 4 processor, 520MB, CentOS-4.2 (Linux kernel 2.6)
52 Automaton Size Condition Factorization Snort-NG No. of states Number of Filtering Rules
53 Effect of Optimizations on Size No. of states LR Tree LR DAG Adaptive Tree Adaptive DAG Adaptive DAG w/ benign non-det Number of Filtering Rules
54 Matching Time Lower Bound Avg. Path Length (in terms of tests) Adaptive Traversal Lower Bound Number of Filtering Rules
55 Matching Time Matching Time (in s) Snort 2 Snort-NG Condition Factorization Number of Filtering Rules
56 Matching Time Matching Time (in s) Snort 2 Snort-NG Condition Factorization Number of Filtering Rules
57 Experiments Setup for Firewall Department firewall rules Firewall rules in the form of iptable rules for a Linux machine Network divided into different subnets 140 filtering rules System: 1.70Ghz pentium 4 processor, 520MB, CentOS-4.2 (Linux kernel 2.6)
58 Automaton Size No. of states 4000 Adaptive Traversal DAG Number of Filtering Rules
59 Matching Time Lower Bound Avg. Path Length (in terms of tests) Lower Bound Actual Path Length Number of Filtering Rules
60 Extending Our Techniques for Content Matching Key Idea Use boolean variables corresponding to strings being matched Test boolean variables to check presence of corresponding string in payload Treat tests on these boolean variables just like tests on other packet fields F 1 : (tcp sport = 80) (content = Command complete ) F 2 : (tcp sport = 80) (content = Bad command ) (content = Bad filename ) F 3 : (tcp sport = 25) (content = Command complete )
61 Extending Our Techniques for Content Matching Key Idea Use boolean variables corresponding to strings being matched Test boolean variables to check presence of corresponding string in payload Treat tests on these boolean variables just like tests on other packet fields F 1 : (tcp sport = 80) (content = Command complete ) F 2 : (tcp sport = 80) (content = Bad command ) (content = Bad filename ) F 3 : (tcp sport = 25) (content = Command complete ) F 1 : C 1 (X 1 = 1) F 2 : C 2 (X 2 = 1) (X 3 = 1) F 3 : C 3 (X 1 = 1)
62 Interesting Questions When to Perform String Matching? Perform string matching before passing packet to packet classification automata Lazy evaluation perform string matching only when packet classification automata can not proceed How to Handle Regular Expressions? Use combined packet-field and string matching as prefilter for RE matching How can we include parts of RE in string matching to get maximize gains from prefiltering?
63 Related Work Left-to-right traversal based techniques PathFinder, DPF share common prefix BPF+ uses global data flow techniques to eliminate redundant tests Can not reason about semantic redundancies in presence of complex test Adaptive traversal Sekar et al, Adaptive Binary Matching [Gustafsson] Do not handle inequalities, disequalities, bit-fields Automata has exponential worst case space complexity Linears size guarded sequential automata require runtime manipulation of match sets Dynamic reordering techniques DPF, Al-Shaer et al maintain statistics regarding traffic Techniques for routers work on fixed number of fields Srinivasan et al, Lakshman et al multidimensional searching problem Woo et al, Gupta et al decision tree based techniques
64 Conclusion and Future Work Summary Developed a new technique for fast packet classification Flexible support diverse applications in a uniform framework Promotes sharing of tests Developed novel techniques for generating packet classification automata that Have polynomial size Virtually constant matching time Demonstrated the gains from our technique for intrusion detection systems and firewalls Future Work Complete the integration and then evaluate the combined content matching operation and packet field matching
65 Thank You Acknowledgement: Sreenath Vasudevan Questions?
66 Computing Match and Candidate Sets P s denotes the conjunction of tests on the path from the start state to s Maintain only the residuals of the original filters in C s and M s with respect to P s
67 Computing Match and Candidate Sets P s denotes the conjunction of tests on the path from the start state to s Maintain only the residuals of the original filters in C s and M s with respect to P s Match Set M 1 = {M F/P s (M = true)} M 2 = {M M 1 M F/P s Pri(M ) > Pri(M)} M s is obtained by considering filters with equal priorities in M 2, and deleting all but one of them.
68 Computing Match and Candidate Sets P s denotes the conjunction of tests on the path from the start state to s Maintain only the residuals of the original filters in C s and M s with respect to P s Candidate Set C(F, M) = {C F M M with Pri(M ) Pri(C)} C s = C(F/P s, M s )
CS 5114: Theory of Algorithms. Tractable Problems. Tractable Problems (cont) Decision Problems. Clifford A. Shaffer. Spring 2014
Department of Computer Science Virginia Tech Blacksburg, Virginia Copyright c 2014 by Clifford A. Shaffer : Theory of Algorithms Title page : Theory of Algorithms Clifford A. Shaffer Spring 2014 Clifford
More informationCS 5114: Theory of Algorithms
CS 5114: Theory of Algorithms Clifford A. Shaffer Department of Computer Science Virginia Tech Blacksburg, Virginia Spring 2014 Copyright c 2014 by Clifford A. Shaffer CS 5114: Theory of Algorithms Spring
More informationComputational Complexity and Intractability: An Introduction to the Theory of NP. Chapter 9
1 Computational Complexity and Intractability: An Introduction to the Theory of NP Chapter 9 2 Objectives Classify problems as tractable or intractable Define decision problems Define the class P Define
More informationChapter 3 Deterministic planning
Chapter 3 Deterministic planning In this chapter we describe a number of algorithms for solving the historically most important and most basic type of planning problem. Two rather strong simplifying assumptions
More informationClasses of Boolean Functions
Classes of Boolean Functions Nader H. Bshouty Eyal Kushilevitz Abstract Here we give classes of Boolean functions that considered in COLT. Classes of Functions Here we introduce the basic classes of functions
More informationMore on NP and Reductions
Indian Institute of Information Technology Design and Manufacturing, Kancheepuram Chennai 600 127, India An Autonomous Institute under MHRD, Govt of India http://www.iiitdm.ac.in COM 501 Advanced Data
More informationIntroduction to Kleene Algebras
Introduction to Kleene Algebras Riccardo Pucella Basic Notions Seminar December 1, 2005 Introduction to Kleene Algebras p.1 Idempotent Semirings An idempotent semiring is a structure S = (S, +,, 1, 0)
More informationIntroduction to Complexity Theory
Introduction to Complexity Theory Read K & S Chapter 6. Most computational problems you will face your life are solvable (decidable). We have yet to address whether a problem is easy or hard. Complexity
More informationEECS 219C: Computer-Aided Verification Boolean Satisfiability Solving III & Binary Decision Diagrams. Sanjit A. Seshia EECS, UC Berkeley
EECS 219C: Computer-Aided Verification Boolean Satisfiability Solving III & Binary Decision Diagrams Sanjit A. Seshia EECS, UC Berkeley Acknowledgments: Lintao Zhang Announcement Project proposals due
More informationNP-Completeness and Boolean Satisfiability
NP-Completeness and Boolean Satisfiability Mridul Aanjaneya Stanford University August 14, 2012 Mridul Aanjaneya Automata Theory 1/ 49 Time-Bounded Turing Machines A Turing Machine that, given an input
More informationTimo Latvala. March 7, 2004
Reactive Systems: Safety, Liveness, and Fairness Timo Latvala March 7, 2004 Reactive Systems: Safety, Liveness, and Fairness 14-1 Safety Safety properties are a very useful subclass of specifications.
More informationLecture Notes On THEORY OF COMPUTATION MODULE -1 UNIT - 2
BIJU PATNAIK UNIVERSITY OF TECHNOLOGY, ODISHA Lecture Notes On THEORY OF COMPUTATION MODULE -1 UNIT - 2 Prepared by, Dr. Subhendu Kumar Rath, BPUT, Odisha. UNIT 2 Structure NON-DETERMINISTIC FINITE AUTOMATA
More informationChapter 2. Reductions and NP. 2.1 Reductions Continued The Satisfiability Problem (SAT) SAT 3SAT. CS 573: Algorithms, Fall 2013 August 29, 2013
Chapter 2 Reductions and NP CS 573: Algorithms, Fall 2013 August 29, 2013 2.1 Reductions Continued 2.1.1 The Satisfiability Problem SAT 2.1.1.1 Propositional Formulas Definition 2.1.1. Consider a set of
More informationAlgorithms. NP -Complete Problems. Dong Kyue Kim Hanyang University
Algorithms NP -Complete Problems Dong Kyue Kim Hanyang University dqkim@hanyang.ac.kr The Class P Definition 13.2 Polynomially bounded An algorithm is said to be polynomially bounded if its worst-case
More informationAutomated Program Verification and Testing 15414/15614 Fall 2016 Lecture 3: Practical SAT Solving
Automated Program Verification and Testing 15414/15614 Fall 2016 Lecture 3: Practical SAT Solving Matt Fredrikson mfredrik@cs.cmu.edu October 17, 2016 Matt Fredrikson SAT Solving 1 / 36 Review: Propositional
More informationNP Completeness and Approximation Algorithms
Winter School on Optimization Techniques December 15-20, 2016 Organized by ACMU, ISI and IEEE CEDA NP Completeness and Approximation Algorithms Susmita Sur-Kolay Advanced Computing and Microelectronic
More informationA An Overview of Complexity Theory for the Algorithm Designer
A An Overview of Complexity Theory for the Algorithm Designer A.1 Certificates and the class NP A decision problem is one whose answer is either yes or no. Two examples are: SAT: Given a Boolean formula
More informationDescription Logics: an Introductory Course on a Nice Family of Logics. Day 2: Tableau Algorithms. Uli Sattler
Description Logics: an Introductory Course on a Nice Family of Logics Day 2: Tableau Algorithms Uli Sattler 1 Warm up Which of the following subsumptions hold? r some (A and B) is subsumed by r some A
More informationAutomata Theory for Presburger Arithmetic Logic
Automata Theory for Presburger Arithmetic Logic References from Introduction to Automata Theory, Languages & Computation and Constraints in Computational Logic Theory & Application Presented by Masood
More informationNP-Completeness. Algorithmique Fall semester 2011/12
NP-Completeness Algorithmique Fall semester 2011/12 1 What is an Algorithm? We haven t really answered this question in this course. Informally, an algorithm is a step-by-step procedure for computing a
More informationCS 154, Lecture 2: Finite Automata, Closure Properties Nondeterminism,
CS 54, Lecture 2: Finite Automata, Closure Properties Nondeterminism, Why so Many Models? Streaming Algorithms 0 42 Deterministic Finite Automata Anatomy of Deterministic Finite Automata transition: for
More informationLecture 7: The Satisfiability Problem
Lecture 7: The Satisfiability Problem 1 Satisfiability 1.1 Classification of Formulas Remember the 2 classifications of problems we have discussed in the past: Satisfiable and Valid. The Classification
More informationTemporal logics and explicit-state model checking. Pierre Wolper Université de Liège
Temporal logics and explicit-state model checking Pierre Wolper Université de Liège 1 Topics to be covered Introducing explicit-state model checking Finite automata on infinite words Temporal Logics and
More informationLecture 25: Cook s Theorem (1997) Steven Skiena. skiena
Lecture 25: Cook s Theorem (1997) Steven Skiena Department of Computer Science State University of New York Stony Brook, NY 11794 4400 http://www.cs.sunysb.edu/ skiena Prove that Hamiltonian Path is NP
More informationTuring Machines and Time Complexity
Turing Machines and Time Complexity Turing Machines Turing Machines (Infinitely long) Tape of 1 s and 0 s Turing Machines (Infinitely long) Tape of 1 s and 0 s Able to read and write the tape, and move
More informationData Mining and Machine Learning
Data Mining and Machine Learning Concept Learning and Version Spaces Introduction Concept Learning Generality Relations Refinement Operators Structured Hypothesis Spaces Simple algorithms Find-S Find-G
More informationComp487/587 - Boolean Formulas
Comp487/587 - Boolean Formulas 1 Logic and SAT 1.1 What is a Boolean Formula Logic is a way through which we can analyze and reason about simple or complicated events. In particular, we are interested
More informationComplexity Theory VU , SS The Polynomial Hierarchy. Reinhard Pichler
Complexity Theory Complexity Theory VU 181.142, SS 2018 6. The Polynomial Hierarchy Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität Wien 15 May, 2018 Reinhard
More informationOutline. Complexity Theory EXACT TSP. The Class DP. Definition. Problem EXACT TSP. Complexity of EXACT TSP. Proposition VU 181.
Complexity Theory Complexity Theory Outline Complexity Theory VU 181.142, SS 2018 6. The Polynomial Hierarchy Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität
More informationAutomata Theory CS Complexity Theory I: Polynomial Time
Automata Theory CS411-2015-17 Complexity Theory I: Polynomial Time David Galles Department of Computer Science University of San Francisco 17-0: Tractable vs. Intractable If a problem is recursive, then
More informationCS256/Spring 2008 Lecture #11 Zohar Manna. Beyond Temporal Logics
CS256/Spring 2008 Lecture #11 Zohar Manna Beyond Temporal Logics Temporal logic expresses properties of infinite sequences of states, but there are interesting properties that cannot be expressed, e.g.,
More informationSummer School on Introduction to Algorithms and Optimization Techniques July 4-12, 2017 Organized by ACMU, ISI and IEEE CEDA.
Summer School on Introduction to Algorithms and Optimization Techniques July 4-12, 2017 Organized by ACMU, ISI and IEEE CEDA NP Completeness Susmita Sur-Kolay Advanced Computing and Microelectronics Unit
More informationNondeterminism. September 7, Nondeterminism
September 7, 204 Introduction is a useful concept that has a great impact on the theory of computation Introduction is a useful concept that has a great impact on the theory of computation So far in our
More informationNondeterministic Finite Automata
Nondeterministic Finite Automata Mahesh Viswanathan Introducing Nondeterminism Consider the machine shown in Figure. Like a DFA it has finitely many states and transitions labeled by symbols from an input
More informationQuIDD-Optimised Quantum Algorithms
QuIDD-Optimised Quantum Algorithms by S K University of York Computer science 3 rd year project Supervisor: Prof Susan Stepney 03/05/2004 1 Project Objectives Investigate the QuIDD optimisation techniques
More informationAlgebraic Dynamic Programming. Solving Satisfiability with ADP
Algebraic Dynamic Programming Session 12 Solving Satisfiability with ADP Robert Giegerich (Lecture) Stefan Janssen (Exercises) Faculty of Technology Summer 2013 http://www.techfak.uni-bielefeld.de/ags/pi/lehre/adp
More informationTheory of Computation p.1/?? Theory of Computation p.2/?? Unknown: Implicitly a Boolean variable: true if a word is
Abstraction of Problems Data: abstracted as a word in a given alphabet. Σ: alphabet, a finite, non-empty set of symbols. Σ : all the words of finite length built up using Σ: Conditions: abstracted as a
More informationSAT Solvers: Theory and Practice
Summer School on Verification Technology, Systems & Applications, September 17, 2008 p. 1/98 SAT Solvers: Theory and Practice Clark Barrett barrett@cs.nyu.edu New York University Summer School on Verification
More informationP is the class of problems for which there are algorithms that solve the problem in time O(n k ) for some constant k.
Complexity Theory Problems are divided into complexity classes. Informally: So far in this course, almost all algorithms had polynomial running time, i.e., on inputs of size n, worst-case running time
More informationThe efficiency of identifying timed automata and the power of clocks
The efficiency of identifying timed automata and the power of clocks Sicco Verwer a,b,1,, Mathijs de Weerdt b, Cees Witteveen b a Eindhoven University of Technology, Department of Mathematics and Computer
More informationReduced Ordered Binary Decision Diagrams
Reduced Ordered Binary Decision Diagrams Lecture #12 of Advanced Model Checking Joost-Pieter Katoen Lehrstuhl 2: Software Modeling & Verification E-mail: katoen@cs.rwth-aachen.de December 13, 2016 c JPK
More informationLecture 1 : Data Compression and Entropy
CPS290: Algorithmic Foundations of Data Science January 8, 207 Lecture : Data Compression and Entropy Lecturer: Kamesh Munagala Scribe: Kamesh Munagala In this lecture, we will study a simple model for
More informationTutorial 6. By:Aashmeet Kalra
Tutorial 6 By:Aashmeet Kalra AGENDA Candidate Elimination Algorithm Example Demo of Candidate Elimination Algorithm Decision Trees Example Demo of Decision Trees Concept and Concept Learning A Concept
More informationFinite Automata. Seungjin Choi
Finite Automata Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr 1 / 28 Outline
More informationLecture Notes on Emptiness Checking, LTL Büchi Automata
15-414: Bug Catching: Automated Program Verification Lecture Notes on Emptiness Checking, LTL Büchi Automata Matt Fredrikson André Platzer Carnegie Mellon University Lecture 18 1 Introduction We ve seen
More informationComputer Science 385 Analysis of Algorithms Siena College Spring Topic Notes: Limitations of Algorithms
Computer Science 385 Analysis of Algorithms Siena College Spring 2011 Topic Notes: Limitations of Algorithms We conclude with a discussion of the limitations of the power of algorithms. That is, what kinds
More informationAn Alternative Construction in Symbolic Reachability Analysis of Second Order Pushdown Systems
An Alternative Construction in Symbolic Reachability Analysis of Second Order Pushdown Systems Anil Seth CSE Department, I.I.T. Kanpur, Kanpur 208016, INDIA. seth@cse.iitk.ac.in Abstract. Recently, it
More informationSAT-Solving: From Davis- Putnam to Zchaff and Beyond Day 3: Recent Developments. Lintao Zhang
SAT-Solving: From Davis- Putnam to Zchaff and Beyond Day 3: Recent Developments Requirements for SAT solvers in the Real World Fast & Robust Given a problem instance, we want to solve it quickly Reliable
More informationINAPPROX APPROX PTAS. FPTAS Knapsack P
CMPSCI 61: Recall From Last Time Lecture 22 Clique TSP INAPPROX exists P approx alg for no ε < 1 VertexCover MAX SAT APPROX TSP some but not all ε< 1 PTAS all ε < 1 ETSP FPTAS Knapsack P poly in n, 1/ε
More informationTheory of computation: initial remarks (Chapter 11)
Theory of computation: initial remarks (Chapter 11) For many purposes, computation is elegantly modeled with simple mathematical objects: Turing machines, finite automata, pushdown automata, and such.
More informationDesign of Distributed Systems Melinda Tóth, Zoltán Horváth
Design of Distributed Systems Melinda Tóth, Zoltán Horváth Design of Distributed Systems Melinda Tóth, Zoltán Horváth Publication date 2014 Copyright 2014 Melinda Tóth, Zoltán Horváth Supported by TÁMOP-412A/1-11/1-2011-0052
More informationA Symbolic Decision Procedure for Symbolic Alternating Finite Automata
MFPS 2010 A Symbolic Decision Procedure for Symbolic Alternating Finite Automata Loris D Antoni 1 Zachary Kincaid 2 Fang Wang 1 1 University of Wisconsin-Madison 2 Princeton University Abstract We introduce
More informationSAT, NP, NP-Completeness
CS 473: Algorithms, Spring 2018 SAT, NP, NP-Completeness Lecture 22 April 13, 2018 Most slides are courtesy Prof. Chekuri Ruta (UIUC) CS473 1 Spring 2018 1 / 57 Part I Reductions Continued Ruta (UIUC)
More informationKB Agents and Propositional Logic
Plan Knowledge-Based Agents Logics Propositional Logic KB Agents and Propositional Logic Announcements Assignment2 mailed out last week. Questions? Knowledge-Based Agents So far, what we ve done is look
More informationAn Introduction to SAT Solving
An Introduction to SAT Solving Applied Logic for Computer Science UWO December 3, 2017 Applied Logic for Computer Science An Introduction to SAT Solving UWO December 3, 2017 1 / 46 Plan 1 The Boolean satisfiability
More informationDETECTION AND REMOVAL OF FIREWALL MISCONFIGURATION
DETECTION AND REMOVAL OF FIREWALL MISCONFIGURATION Frédéric Cuppens Nora Cuppens-Boulahia Joaquín García-Alfaro GET/ENST-Bretagne, 2, rue de la Châtaigneraie, 35576 Cesson Sévigné - France {frederic.cuppens,nora.cuppens}@enst-bretagne.fr
More informationCISC 4090: Theory of Computation Chapter 1 Regular Languages. Section 1.1: Finite Automata. What is a computer? Finite automata
CISC 4090: Theory of Computation Chapter Regular Languages Xiaolan Zhang, adapted from slides by Prof. Werschulz Section.: Finite Automata Fordham University Department of Computer and Information Sciences
More informationA brief introduction to Logic. (slides from
A brief introduction to Logic (slides from http://www.decision-procedures.org/) 1 A Brief Introduction to Logic - Outline Propositional Logic :Syntax Propositional Logic :Semantics Satisfiability and validity
More informationan efficient procedure for the decision problem. We illustrate this phenomenon for the Satisfiability problem.
1 More on NP In this set of lecture notes, we examine the class NP in more detail. We give a characterization of NP which justifies the guess and verify paradigm, and study the complexity of solving search
More informationFinite Automata - Deterministic Finite Automata. Deterministic Finite Automaton (DFA) (or Finite State Machine)
Finite Automata - Deterministic Finite Automata Deterministic Finite Automaton (DFA) (or Finite State Machine) M = (K, Σ, δ, s, A), where K is a finite set of states Σ is an input alphabet s K is a distinguished
More informationLecture 14 - P v.s. NP 1
CME 305: Discrete Mathematics and Algorithms Instructor: Professor Aaron Sidford (sidford@stanford.edu) February 27, 2018 Lecture 14 - P v.s. NP 1 In this lecture we start Unit 3 on NP-hardness and approximation
More informationcse303 ELEMENTS OF THE THEORY OF COMPUTATION Professor Anita Wasilewska
cse303 ELEMENTS OF THE THEORY OF COMPUTATION Professor Anita Wasilewska LECTURE 6 CHAPTER 2 FINITE AUTOMATA 2. Nondeterministic Finite Automata NFA 3. Finite Automata and Regular Expressions 4. Languages
More informationComputability and Complexity Theory: An Introduction
Computability and Complexity Theory: An Introduction meena@imsc.res.in http://www.imsc.res.in/ meena IMI-IISc, 20 July 2006 p. 1 Understanding Computation Kinds of questions we seek answers to: Is a given
More informationDeterministic Finite Automata. Non deterministic finite automata. Non-Deterministic Finite Automata (NFA) Non-Deterministic Finite Automata (NFA)
Deterministic Finite Automata Non deterministic finite automata Automata we ve been dealing with have been deterministic For every state and every alphabet symbol there is exactly one move that the machine
More informationIntractable Problems [HMU06,Chp.10a]
Intractable Problems [HMU06,Chp.10a] Time-Bounded Turing Machines Classes P and NP Polynomial-Time Reductions A 10 Minute Motivation https://www.youtube.com/watch?v=yx40hbahx3s 1 Time-Bounded TM s A Turing
More informationCS4026 Formal Models of Computation
CS4026 Formal Models of Computation Turing Machines Turing Machines Abstract but accurate model of computers Proposed by Alan Turing in 1936 There weren t computers back then! Turing s motivation: find
More informationBBM402-Lecture 11: The Class NP
BBM402-Lecture 11: The Class NP Lecturer: Lale Özkahya Resources for the presentation: http://ocw.mit.edu/courses/electrical-engineering-andcomputer-science/6-045j-automata-computability-andcomplexity-spring-2011/syllabus/
More informationClause/Term Resolution and Learning in the Evaluation of Quantified Boolean Formulas
Journal of Artificial Intelligence Research 1 (1993) 1-15 Submitted 6/91; published 9/91 Clause/Term Resolution and Learning in the Evaluation of Quantified Boolean Formulas Enrico Giunchiglia Massimo
More information(Refer Slide Time: 0:21)
Theory of Computation Prof. Somenath Biswas Department of Computer Science and Engineering Indian Institute of Technology Kanpur Lecture 7 A generalisation of pumping lemma, Non-deterministic finite automata
More informationFinite Automata Part Two
Finite Automata Part Two DFAs A DFA is a Deterministic Finite Automaton A DFA is defined relative to some alphabet Σ. For each state in the DFA, there must be exactly one transition defined for each symbol
More informationarxiv: v1 [cs.ds] 9 Apr 2018
From Regular Expression Matching to Parsing Philip Bille Technical University of Denmark phbi@dtu.dk Inge Li Gørtz Technical University of Denmark inge@dtu.dk arxiv:1804.02906v1 [cs.ds] 9 Apr 2018 Abstract
More informationLecture 4 : Quest for Structure in Counting Problems
CS6840: Advanced Complexity Theory Jan 10, 2012 Lecture 4 : Quest for Structure in Counting Problems Lecturer: Jayalal Sarma M.N. Scribe: Dinesh K. Theme: Between P and PSPACE. Lecture Plan:Counting problems
More informationExercises 1 - Solutions
Exercises 1 - Solutions SAV 2013 1 PL validity For each of the following propositional logic formulae determine whether it is valid or not. If it is valid prove it, otherwise give a counterexample. Note
More informationCorrectness of Dijkstra s algorithm
Correctness of Dijkstra s algorithm Invariant: When vertex u is deleted from the priority queue, d[u] is the correct length of the shortest path from the source s to vertex u. Additionally, the value d[u]
More informationThe Class NP. NP is the problems that can be solved in polynomial time by a nondeterministic machine.
The Class NP NP is the problems that can be solved in polynomial time by a nondeterministic machine. NP The time taken by nondeterministic TM is the length of the longest branch. The collection of all
More informationFinite Automata. Mahesh Viswanathan
Finite Automata Mahesh Viswanathan In this lecture, we will consider different models of finite state machines and study their relative power. These notes assume that the reader is familiar with DFAs,
More informationNP-Completeness. Andreas Klappenecker. [based on slides by Prof. Welch]
NP-Completeness Andreas Klappenecker [based on slides by Prof. Welch] 1 Prelude: Informal Discussion (Incidentally, we will never get very formal in this course) 2 Polynomial Time Algorithms Most of the
More informationExam Computability and Complexity
Total number of points:... Number of extra sheets of paper:... Exam Computability and Complexity by Jiri Srba, January 2009 Student s full name CPR number Study number Before you start, fill in the three
More informationFuzzy Answer Set semantics for Residuated Logic programs
semantics for Logic Nicolás Madrid & Universidad de Málaga September 23, 2009 Aims of this paper We are studying the introduction of two kinds of negations into residuated : Default negation: This negation
More informationCritical Reading of Optimization Methods for Logical Inference [1]
Critical Reading of Optimization Methods for Logical Inference [1] Undergraduate Research Internship Department of Management Sciences Fall 2007 Supervisor: Dr. Miguel Anjos UNIVERSITY OF WATERLOO Rajesh
More informationPattern Recognition and Machine Learning. Learning and Evaluation of Pattern Recognition Processes
Pattern Recognition and Machine Learning James L. Crowley ENSIMAG 3 - MMIS Fall Semester 2016 Lesson 1 5 October 2016 Learning and Evaluation of Pattern Recognition Processes Outline Notation...2 1. The
More informationGraduate Algorithms CS F-21 NP & Approximation Algorithms
Graduate Algorithms CS673-2016F-21 NP & Approximation Algorithms David Galles Department of Computer Science University of San Francisco 21-0: Classes of Problems Consider three problem classes: Polynomial
More informationU.C. Berkeley CS278: Computational Complexity Professor Luca Trevisan August 30, Notes for Lecture 1
U.C. Berkeley CS278: Computational Complexity Handout N1 Professor Luca Trevisan August 30, 2004 Notes for Lecture 1 This course assumes CS170, or equivalent, as a prerequisite. We will assume that the
More informationThe algorithmic analysis of hybrid system
The algorithmic analysis of hybrid system Authors: R.Alur, C. Courcoubetis etc. Course teacher: Prof. Ugo Buy Xin Li, Huiyong Xiao Nov. 13, 2002 Summary What s a hybrid system? Definition of Hybrid Automaton
More informationUnranked Tree Automata with Sibling Equalities and Disequalities
Unranked Tree Automata with Sibling Equalities and Disequalities Wong Karianto Christof Löding Lehrstuhl für Informatik 7, RWTH Aachen, Germany 34th International Colloquium, ICALP 2007 Xu Gao (NFS) Unranked
More informationEssential facts about NP-completeness:
CMPSCI611: NP Completeness Lecture 17 Essential facts about NP-completeness: Any NP-complete problem can be solved by a simple, but exponentially slow algorithm. We don t have polynomial-time solutions
More informationIntractable Problems. Time-Bounded Turing Machines Classes P and NP Polynomial-Time Reductions
Intractable Problems Time-Bounded Turing Machines Classes P and NP Polynomial-Time Reductions 1 Time-Bounded TM s A Turing machine that, given an input of length n, always halts within T(n) moves is said
More information22c:145 Artificial Intelligence
22c:145 Artificial Intelligence Fall 2005 Propositional Logic Cesare Tinelli The University of Iowa Copyright 2001-05 Cesare Tinelli and Hantao Zhang. a a These notes are copyrighted material and may not
More informationDecision Tree Learning Lecture 2
Machine Learning Coms-4771 Decision Tree Learning Lecture 2 January 28, 2008 Two Types of Supervised Learning Problems (recap) Feature (input) space X, label (output) space Y. Unknown distribution D over
More informationLOGIC PROPOSITIONAL REASONING
LOGIC PROPOSITIONAL REASONING WS 2017/2018 (342.208) Armin Biere Martina Seidl biere@jku.at martina.seidl@jku.at Institute for Formal Models and Verification Johannes Kepler Universität Linz Version 2018.1
More informationNondeterministic finite automata
Lecture 3 Nondeterministic finite automata This lecture is focused on the nondeterministic finite automata (NFA) model and its relationship to the DFA model. Nondeterminism is an important concept in the
More informationPart V. Intractable Problems
Part V Intractable Problems 507 Chapter 16 N P-Completeness Up to now, we have focused on developing efficient algorithms for solving problems. The word efficient is somewhat subjective, and the degree
More informationReduced Ordered Binary Decision Diagram with Implied Literals: A New knowledge Compilation Approach
Reduced Ordered Binary Decision Diagram with Implied Literals: A New knowledge Compilation Approach Yong Lai, Dayou Liu, Shengsheng Wang College of Computer Science and Technology Jilin University, Changchun
More informationA Lower Bound of 2 n Conditional Jumps for Boolean Satisfiability on A Random Access Machine
A Lower Bound of 2 n Conditional Jumps for Boolean Satisfiability on A Random Access Machine Samuel C. Hsieh Computer Science Department, Ball State University July 3, 2014 Abstract We establish a lower
More informationThe Complexity of Lattice-Based Fuzzy Description Logics
Noname manuscript No (will be inserted by the editor) The Complexity of Lattice-Based Fuzzy Description Logics From ALC to SHI Stefan Borgwardt Rafael Peñaloza the date of receipt and acceptance should
More informationIn complexity theory, algorithms and problems are classified by the growth order of computation time as a function of instance size.
10 2.2. CLASSES OF COMPUTATIONAL COMPLEXITY An optimization problem is defined as a class of similar problems with different input parameters. Each individual case with fixed parameter values is called
More informationCS 154, Lecture 3: DFA NFA, Regular Expressions
CS 154, Lecture 3: DFA NFA, Regular Expressions Homework 1 is coming out Deterministic Finite Automata Computation with finite memory Non-Deterministic Finite Automata Computation with finite memory and
More informationPart 1: Propositional Logic
Part 1: Propositional Logic Literature (also for first-order logic) Schöning: Logik für Informatiker, Spektrum Fitting: First-Order Logic and Automated Theorem Proving, Springer 1 Last time 1.1 Syntax
More informationOverview. Discrete Event Systems Verification of Finite Automata. What can finite automata be used for? What can finite automata be used for?
Computer Engineering and Networks Overview Discrete Event Systems Verification of Finite Automata Lothar Thiele Introduction Binary Decision Diagrams Representation of Boolean Functions Comparing two circuits
More information[read Chapter 2] [suggested exercises 2.2, 2.3, 2.4, 2.6] General-to-specific ordering over hypotheses
1 CONCEPT LEARNING AND THE GENERAL-TO-SPECIFIC ORDERING [read Chapter 2] [suggested exercises 2.2, 2.3, 2.4, 2.6] Learning from examples General-to-specific ordering over hypotheses Version spaces and
More information