Arithmetic Complexity in Learning Theory
|
|
|
- Jane Hoover
- 5 years ago
- Views:
Transcription
1 Arithmetic Complexity in Learning Theory Achilles Beros Department of Mathematics, UW - Madison March 25, 2011
2 Background
3 Learning Theory Background
4 Background Learning Theory Different models of learning: EX, BC, FIN, etc.
5 Background Learning Theory Different models of learning: EX, BC, FIN, etc. In each, a learner is being fed an enumeration and is trying to identify the content
6 Background Learning Theory Different models of learning: EX, BC, FIN, etc. In each, a learner is being fed an enumeration and is trying to identify the content Hypotheses are codes for c.e. sets (or recursive functions)
7 Background Learning Theory Different models of learning: EX, BC, FIN, etc. In each, a learner is being fed an enumeration and is trying to identify the content Hypotheses are codes for c.e. sets (or recursive functions) TxtEx-learning
8 Background Learning Theory Different models of learning: EX, BC, FIN, etc. In each, a learner is being fed an enumeration and is trying to identify the content Hypotheses are codes for c.e. sets (or recursive functions) TxtEx-learning Learning from text, i.e. arbitrary enumeration
9 Background Learning Theory Different models of learning: EX, BC, FIN, etc. In each, a learner is being fed an enumeration and is trying to identify the content Hypotheses are codes for c.e. sets (or recursive functions) TxtEx-learning Learning from text, i.e. arbitrary enumeration A learner TxtEx-learns a set if its output eventually stabilizes to a correct hypothesis on every text
10 Background Learning Theory Different models of learning: EX, BC, FIN, etc. In each, a learner is being fed an enumeration and is trying to identify the content Hypotheses are codes for c.e. sets (or recursive functions) TxtEx-learning Learning from text, i.e. arbitrary enumeration A learner TxtEx-learns a set if its output eventually stabilizes to a correct hypothesis on every text Learns a family if it can learn each set
11 Background
12 Examples Background
13 Background Examples A singleton is TxtEx-learnable
14 Background Examples A singleton is TxtEx-learnable Finite families is TxtEx-learnable
15 Background Examples A singleton is TxtEx-learnable Finite families is TxtEx-learnable Family of finite sets is TxtEx-learnable
16 Background Examples A singleton is TxtEx-learnable Finite families is TxtEx-learnable Family of finite sets is TxtEx-learnable All finite sets and one infinite set is not TxtEx-learnable
17 Background Examples A singleton is TxtEx-learnable Finite families is TxtEx-learnable Family of finite sets is TxtEx-learnable All finite sets and one infinite set is not TxtEx-learnable F = {H 0, L 0, H 1, L 1,...}, where H e = {e + x : x W e } and L e = {e + x : x N} is not TxtEx-learnable
18
19 Complexity
20 Complexity The goal is to characterize the arithmetic complexity of learning criteria
21 Complexity The goal is to characterize the arithmetic complexity of learning criteria In order to get reasonable upper bounds, we only examine u.c.e. families
22 Complexity The goal is to characterize the arithmetic complexity of learning criteria In order to get reasonable upper bounds, we only examine u.c.e. families EXL denotes the set of Σ 0 1 codes for u.c.e. families that are TxtEx-learnable
23 Complexity The goal is to characterize the arithmetic complexity of learning criteria In order to get reasonable upper bounds, we only examine u.c.e. families EXL denotes the set of Σ 0 1 codes for u.c.e. families that are TxtEx-learnable BCL denotes the set of Σ 0 1 codes for u.c.e. families that are TxtBC-learnable
24 Complexity The goal is to characterize the arithmetic complexity of learning criteria In order to get reasonable upper bounds, we only examine u.c.e. families EXL denotes the set of Σ 0 1 codes for u.c.e. families that are TxtEx-learnable BCL denotes the set of Σ 0 1 codes for u.c.e. families that are TxtBC-learnable EXL denotes the set of Σ 0 1 codes for u.c.e families that are TxtEx -learnable
25
26 Results so far
27 Results so far BCL is Π 0 4 -hard
28 Results so far BCL is Π 0 4 -hard BCL is Σ 0 5
29 Results so far BCL is Π 0 4 -hard BCL is Σ 0 5 EXL is Π 0 4 -hard
30 Results so far BCL is Π 0 4 -hard BCL is Σ 0 5 EXL is Π 0 4 -hard EXL is Σ 0 4 -complete
31
32 Theorem EXL is Σ 0 4 -complete.
33 Theorem EXL is Σ 0 4 -complete. Two facts must be proved
34 Theorem EXL is Σ 0 4 -complete. Two facts must be proved Lemma 1 EXL is Σ 0 4
35 Theorem EXL is Σ 0 4 -complete. Two facts must be proved Lemma 1 EXL is Σ 0 4 Lemma 2 EXL is Σ 0 4 -hard
36 Theorem EXL is Σ 0 4 -complete. Two facts must be proved Lemma 1 EXL is Σ 0 4 Lemma 2 EXL is Σ 0 4 -hard Lemma 1 can be proved through a routine syntactic description of TxtEx-learning
37 Theorem EXL is Σ 0 4 -complete. Two facts must be proved Lemma 1 EXL is Σ 0 4 Lemma 2 EXL is Σ 0 4 -hard Lemma 1 can be proved through a routine syntactic description of TxtEx-learning e EXL iff k[ i σ, s τ, t[(content(σ) L i,s ) (content(τ) L i,t ) M k (στ) = M(σ)] i, j[l i = W j σ, s τ, t[(content(σ) L i,s ) (content(τ) L i,t M k (στ) j)]]
38
39 Proof of Lemma 2
40 Proof of Lemma 2 As before, let H e = {e + x : x W e }, L e = {e + x : x N}, and F = {L 0, H 0, L 1, H 1,...}.
41 Proof of Lemma 2 As before, let H e = {e + x : x W e }, L e = {e + x : x N}, and F = {L 0, H 0, L 1, H 1,...}. F is u.c.e. with H e as the (2e + 1) st column and L e as the (2e) th column. Let F n denote the n th column.
42 Proof of Lemma 2 As before, let H e = {e + x : x W e }, L e = {e + x : x N}, and F = {L 0, H 0, L 1, H 1,...}. F is u.c.e. with H e as the (2e + 1) st column and L e as the (2e) th column. Let F n denote the n th column. Construct a family G e,x as follows, for x < e:
43 Proof of Lemma 2 As before, let H e = {e + x : x W e }, L e = {e + x : x N}, and F = {L 0, H 0, L 1, H 1,...}. F is u.c.e. with H e as the (2e + 1) st column and L e as the (2e) th column. Let F n denote the n th column. Construct a family G e,x as follows, for x < e: Simultaneously construct R n for x n < e. F n is assigned to R n
44 Proof of Lemma 2 As before, let H e = {e + x : x W e }, L e = {e + x : x N}, and F = {L 0, H 0, L 1, H 1,...}. F is u.c.e. with H e as the (2e + 1) st column and L e as the (2e) th column. Let F n denote the n th column. Construct a family G e,x as follows, for x < e: Simultaneously construct R n for x n < e. F n is assigned to R n Stage 0: n(r n = )
45 Proof of Lemma 2 As before, let H e = {e + x : x W e }, L e = {e + x : x N}, and F = {L 0, H 0, L 1, H 1,...}. F is u.c.e. with H e as the (2e + 1) st column and L e as the (2e) th column. Let F n denote the n th column. Construct a family G e,x as follows, for x < e: Simultaneously construct R n for x n < e. F n is assigned to R n Stage 0: n(r n = ) Stage s:
46 Proof of Lemma 2 As before, let H e = {e + x : x W e }, L e = {e + x : x N}, and F = {L 0, H 0, L 1, H 1,...}. F is u.c.e. with H e as the (2e + 1) st column and L e as the (2e) th column. Let F n denote the n th column. Construct a family G e,x as follows, for x < e: Simultaneously construct R n for x n < e. F n is assigned to R n Stage 0: n(r n = ) Stage s: Suppose [i, i + j] has just entered We,s
47 Proof of Lemma 2 As before, let H e = {e + x : x W e }, L e = {e + x : x N}, and F = {L 0, H 0, L 1, H 1,...}. F is u.c.e. with H e as the (2e + 1) st column and L e as the (2e) th column. Let F n denote the n th column. Construct a family G e,x as follows, for x < e: Simultaneously construct R n for x n < e. F n is assigned to R n Stage 0: n(r n = ) Stage s: Suppose [i, i + j] has just entered We,s Compute Fn,j and enumerate the result into the i th column of R n
48 Proof of Lemma 2 As before, let H e = {e + x : x W e }, L e = {e + x : x N}, and F = {L 0, H 0, L 1, H 1,...}. F is u.c.e. with H e as the (2e + 1) st column and L e as the (2e) th column. Let F n denote the n th column. Construct a family G e,x as follows, for x < e: Simultaneously construct R n for x n < e. F n is assigned to R n Stage 0: n(r n = ) Stage s: Suppose [i, i + j] has just entered We,s Compute Fn,j and enumerate the result into the i th column of R n Enumerate n0 = n/2 into the i th column of R n
49 Proof of Lemma 2 As before, let H e = {e + x : x W e }, L e = {e + x : x N}, and F = {L 0, H 0, L 1, H 1,...}. F is u.c.e. with H e as the (2e + 1) st column and L e as the (2e) th column. Let F n denote the n th column. Construct a family G e,x as follows, for x < e: Simultaneously construct R n for x n < e. F n is assigned to R n Stage 0: n(r n = ) Stage s: Suppose [i, i + j] has just entered We,s Compute Fn,j and enumerate the result into the i th column of R n Enumerate n0 = n/2 into the i th column of R n Put R n,k into the k th column of R n for k [i, i + j] k [i,i+j]
50
51 Let G e,x = n [x,e) R n
52 Let G e,x = n [x,e) R n Let g be a computable function so that g(e, x) is a Σ 0 1 code for G e,x
53 Let G e,x = n [x,e) R n Let g be a computable function so that g(e, x) is a Σ 0 1 code for G e,x If e COINF, G e,x is a collection of finite sets
54 Let G e,x = n [x,e) R n Let g be a computable function so that g(e, x) is a Σ 0 1 code for G e,x If e COINF, G e,x is a collection of finite sets If e COF, G e,x will be finite and contain some finite sets and F n, for x n < e
55 Let G e,x = n [x,e) R n Let g be a computable function so that g(e, x) is a Σ 0 1 code for G e,x If e COINF, G e,x is a collection of finite sets If e COF, G e,x will be finite and contain some finite sets and F n, for x n < e Now we define the reduction Fix P, an arbitrary unary Σ 0 4 predicate
56 Let G e,x = n [x,e) R n Let g be a computable function so that g(e, x) is a Σ 0 1 code for G e,x If e COINF, G e,x is a collection of finite sets If e COF, G e,x will be finite and contain some finite sets and F n, for x n < e Now we define the reduction Fix P, an arbitrary unary Σ 0 4 predicate Fix r(x, y) to be computable and one-to-one such that P(x) y[r(x, y) COINF ] and P(x) y[r(x, y) COF ]
57 Let G e,x = n [x,e) R n Let g be a computable function so that g(e, x) is a Σ 0 1 code for G e,x If e COINF, G e,x is a collection of finite sets If e COF, G e,x will be finite and contain some finite sets and F n, for x n < e Now we define the reduction Fix P, an arbitrary unary Σ 0 4 predicate Fix r(x, y) to be computable and one-to-one such that P(x) y[r(x, y) COINF ] and P(x) y[r(x, y) COF ] Define s(x, y) to be computable and one-to-one such that for fixed x, s(x, y) is an increasing subsequence of r(x, y)
58 Let G e,x = n [x,e) R n Let g be a computable function so that g(e, x) is a Σ 0 1 code for G e,x If e COINF, G e,x is a collection of finite sets If e COF, G e,x will be finite and contain some finite sets and F n, for x n < e Now we define the reduction Fix P, an arbitrary unary Σ 0 4 predicate Fix r(x, y) to be computable and one-to-one such that P(x) y[r(x, y) COINF ] and P(x) y[r(x, y) COF ] Define s(x, y) to be computable and one-to-one such that for fixed x, s(x, y) is an increasing subsequence of r(x, y) Define h to be a computable function such that h(e) is a Σ 0 1 code for H e = y N G s(e,y+1),s(e,y)
59
60 Case 1: P(e)
61 Case 1: P(e) For n N there is a y such that s(e, y 1) n < s(e, y)
62 Case 1: P(e) For n N there is a y such that s(e, y 1) n < s(e, y) F n G s(e,y+1),s(e,y), since s(e, y + 1) COF, hence F H e
63 Case 1: P(e) For n N there is a y such that s(e, y 1) n < s(e, y) F n G s(e,y+1),s(e,y), since s(e, y + 1) COF, hence F H e F is not TxtEx-learnable, thus H e is not
64
65 Case 2: P(e)
66 Case 2: P(e) Pick k 0 such that n k 0 [s(e, n) COINF ]
67 Case 2: P(e) Pick k 0 such that n k 0 [s(e, n) COINF ] For n > k 0, G s(e,n+1),s(e,n) is a collection of finite sets which do not contain any numbers less than s(e, k 0 )
68 Case 2: P(e) Pick k 0 such that n k 0 [s(e, n) COINF ] For n > k 0, G s(e,n+1),s(e,n) is a collection of finite sets which do not contain any numbers less than s(e, k 0 ) For n k 0, if A G s(e,n+1),s(e,n), either A is finite or k < s(e, k 0 )(k A)
69 Case 2: P(e) Pick k 0 such that n k 0 [s(e, n) COINF ] For n > k 0, G s(e,n+1),s(e,n) is a collection of finite sets which do not contain any numbers less than s(e, k 0 ) For n k 0, if A G s(e,n+1),s(e,n), either A is finite or k < s(e, k 0 )(k A) Fix M 0 that learns the finite family {F 0,..., F s(e,k0 )}
70 Case 2: P(e) Pick k 0 such that n k 0 [s(e, n) COINF ] For n > k 0, G s(e,n+1),s(e,n) is a collection of finite sets which do not contain any numbers less than s(e, k 0 ) For n k 0, if A G s(e,n+1),s(e,n), either A is finite or k < s(e, k 0 )(k A) Fix M 0 that learns the finite family {F 0,..., F s(e,k0 )} Fix M 1 that learns the collection of all finite sets
71 Case 2: P(e) Pick k 0 such that n k 0 [s(e, n) COINF ] For n > k 0, G s(e,n+1),s(e,n) is a collection of finite sets which do not contain any numbers less than s(e, k 0 ) For n k 0, if A G s(e,n+1),s(e,n), either A is finite or k < s(e, k 0 )(k A) Fix M 0 that learns the finite family {F 0,..., F s(e,k0 )} Fix M 1 that learns the collection of all finite sets { M 0 (σ) if k < s(e, k 0 )(k content(σ)) M(σ) = M 1 (σ) otherwise TxtEx-learns H e
72 Thank You
On the non-existence of maximal inference degrees for language identification
On the non-existence of maximal inference degrees for language identification Sanjay Jain Institute of Systems Science National University of Singapore Singapore 0511 Republic of Singapore sanjay@iss.nus.sg
Chapter 6. Hypothesis Tests Lecture 20: UMP tests and Neyman-Pearson lemma
Chapter 6. Hypothesis Tests Lecture 20: UMP tests and Neyman-Pearson lemma Theory of testing hypotheses X: a sample from a population P in P, a family of populations. Based on the observed X, we test a
Iterative Learning from Texts and Counterexamples using Additional Information
Sacred Heart University DigitalCommons@SHU Computer Science & Information Technology Faculty Publications Computer Science & Information Technology 2011 Iterative Learning from Texts and Counterexamples
Inductive Inference Systems for Learning Classes of Algorithmically Generated Sets and Structures Valentina S. Harizanov
Chapter 2 Inductive Inference Systems for Learning Classes of Algorithmically Generated Sets and Structures Valentina S. Harizanov Abstract. Computability theorists have extensively studied sets whose
INDEX SETS OF UNIVERSAL CODES
INDEX SETS OF UNIVERSAL CODES ACHILLES A. BEROS AND KONSTANTINOS A. BEROS Abstract. We examine sets of codes such that certain properties are invariant under the choice of oracle from a range of possible
Language Learning from Texts: Degrees of Intrinsic Complexity and Their Characterizations
Language Learning from Texts: Degrees of Intrinsic Complexity and Their Characterizations Sanjay Jain School of Computing National University of Singapore 3 Science Drive 2 Singapore 117543 sanjay@comp.nus.edu.sg
Models of Language Acquisition: Part II
Models of Language Acquisition: Part II Matilde Marcolli CS101: Mathematical and Computational Linguistics Winter 2015 Probably Approximately Correct Model of Language Learning General setting of Statistical
Learning via Finitely Many Queries
Learning via Finitely Many Queries Andrew C. Lee University of Louisiana at Lafayette 1 Abstract This work introduces a new query inference model that can access data and communicate with a teacher by
[Read Ch. 5] [Recommended exercises: 5.2, 5.3, 5.4]
![[Read Ch. 5] [Recommended exercises: 5.2, 5.3, 5.4]](/thumbs/87/95366673.jpg "[Read Ch. 5] [Recommended exercises: 5.2, 5.3, 5.4]")
Evaluating Hypotheses [Read Ch. 5] [Recommended exercises: 5.2, 5.3, 5.4] Sample error, true error Condence intervals for observed hypothesis error Estimators Binomial distribution, Normal distribution,
1 The decision problem for First order logic
Math 260A Mathematical Logic Scribe Notes UCSD Winter Quarter 2012 Instructor: Sam Buss Notes by: James Aisenberg April 27th 1 The decision problem for First order logic Fix a finite language L. Define
VC Dimension Review. The purpose of this document is to review VC dimension and PAC learning for infinite hypothesis spaces.
VC Dimension Review The purpose of this document is to review VC dimension and PAC learning for infinite hypothesis spaces. Previously, in discussing PAC learning, we were trying to answer questions about
On the learnability of vector spaces
Journal of Computer and System Sciences 73 (2007) 109 122 www.elsevier.com/locate/jcss On the learnability of vector spaces Valentina S. Harizanov a,1, Frank Stephan b,,2 a Department of Mathematics, George
PAC LEARNING, VC DIMENSION, AND THE ARITHMETIC HIERARCHY
PAC LEARNING, VC DIMENSION, AND THE ARITHMETIC HIERARCHY WESLEY CALVERT Abstract. We compute that the index set of PAC-learnable concept classes is m-complete Σ 0 3 within the set of indices for all concept
Uncountable Automatic Classes and Learning
Uncountable Automatic Classes and Learning Sanjay Jain a,1, Qinglong Luo a, Pavel Semukhin b,2, Frank Stephan c,3 a Department of Computer Science, National University of Singapore, Singapore 117417, Republic
uring Reducibility Dept. of Computer Sc. & Engg., IIT Kharagpur 1 Turing Reducibility
uring Reducibility Dept. of Computer Sc. & Engg., IIT Kharagpur 1 Turing Reducibility uring Reducibility Dept. of Computer Sc. & Engg., IIT Kharagpur 2 FINITE We have already seen that the language FINITE
The Synthesis of Language Learners
The Synthesis of Language Learners Ganesh R. Baliga Computer Science Department Rowan College of New Jersey Mullica Hill, NJ 08024, USA (Email: baliga@gboro.rowan.edu) John Case Department of CIS University
First-Order Logic. 1 Syntax. Domain of Discourse. FO Vocabulary. Terms
First-Order Logic 1 Syntax Domain of Discourse The domain of discourse for first order logic is FO structures or models. A FO structure contains Relations Functions Constants (functions of arity 0) FO
Parallel Learning of Automatic Classes of Languages
Parallel Learning of Automatic Classes of Languages Sanjay Jain a,1, Efim Kinber b a School of Computing, National University of Singapore, Singapore 117417. b Department of Computer Science, Sacred Heart
The Intrinsic Complexity of Language Identification
The Intrinsic Complexity of Language Identification Sanjay Jain Department of Information Systems and Computer Science National University of Singapore Singapore 119260, Republic of Singapore Email: sanjay@iscs.nus.sg
Lecture 28: Generalized Minimum Distance Decoding
Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 007) Lecture 8: Generalized Minimum Distance Decoding November 5, 007 Lecturer: Atri Rudra Scribe: Sandipan Kundu & Atri Rudra 1
Chapter 1 Divide and Conquer Algorithm Theory WS 2016/17 Fabian Kuhn
Chapter 1 Divide and Conquer Algorithm Theory WS 2016/17 Fabian Kuhn Formulation of the D&C principle Divide-and-conquer method for solving a problem instance of size n: 1. Divide n c: Solve the problem
Automata Techniques for Query Inference Machines
Automata Techniques for Query Inference Machines William Gasarch University of Maryland Geoffrey R. Hird Arcot Systems Abstract In prior papers the following question was considered: which classes of computable
Computational Learning Theory (COLT)
Computational Learning Theory (COLT) Goals: Theoretical characterization of 1 Difficulty of machine learning problems Under what conditions is learning possible and impossible? 2 Capabilities of machine
(i) By Corollary 1.3 (a) on P. 88, we know that the predicate. Q(x, y, z) S 1 (x, (z) 1, y, (z) 2 )
COMPUTABILITY: ANSWERS TO SOME EXERCISES IN CHAPTER 5 AND 6 5.1.5, P. 90. 1. This exercise is important for several reasons. For example, it tells us that taking inverses is an effect operation; see Example
Preliminaries. Introduction to EF-games. Inexpressivity results for first-order logic. Normal forms for first-order logic
Introduction to EF-games Inexpressivity results for first-order logic Normal forms for first-order logic Algorithms and complexity for specific classes of structures General complexity bounds Preliminaries
Evaluating Hypotheses
Evaluating Hypotheses IEEE Expert, October 1996 1 Evaluating Hypotheses Sample error, true error Confidence intervals for observed hypothesis error Estimators Binomial distribution, Normal distribution,
AN INTRODUCTION TO COMPUTABILITY THEORY
AN INTRODUCTION TO COMPUTABILITY THEORY CINDY CHUNG Abstract. This paper will give an introduction to the fundamentals of computability theory. Based on Robert Soare s textbook, The Art of Turing Computability:
First Order Logic (FOL) 1 znj/dm2017
First Order Logic (FOL) 1 http://lcs.ios.ac.cn/ znj/dm2017 Naijun Zhan March 19, 2017 1 Special thanks to Profs Hanpin Wang (PKU) and Lijun Zhang (ISCAS) for their courtesy of the slides on this course.
Language Acquisition and Parameters: Part II
Language Acquisition and Parameters: Part II Matilde Marcolli CS0: Mathematical and Computational Linguistics Winter 205 Transition Matrices in the Markov Chain Model absorbing states correspond to local
Computational Learning Theory. Definitions
Computational Learning Theory Computational learning theory is interested in theoretical analyses of the following issues. What is needed to learn effectively? Sample complexity. How many examples? Computational
Learning without Coding
Learning without Coding Sanjay Jain a,1, Samuel E. Moelius III b,, Sandra Zilles c,2 a Department of Computer Science, National University of Singapore, Singapore 117417, Republic of Singapore b IDA Center
Index sets of universal codes
Index sets of universal codes Konstantinos Beros (UNT) joint with Achilles Beros (University of Hawai i) Medvedev reductions Definition Suppose A, B are subsets of 2 ω or families of functions on ω. Say
Logic: The Big Picture
Logic: The Big Picture A typical logic is described in terms of syntax: what are the legitimate formulas semantics: under what circumstances is a formula true proof theory/ axiomatization: rules for proving
The Arithmetical Hierarchy
Chapter 11 The Arithmetical Hierarchy Think of K as posing the problem of induction for computational devices, for it is impossible to tell for sure whether a given computation will never halt. Thus, K
The L Machines are very high-level, in two senses:
What is a Computer? State of the machine. CMPSCI 630: Programming Languages An Abstract Machine for Control Spring 2009 (with thanks to Robert Harper) Internal registers, memory, etc. Initial and final
Final Exam (100 points)
Final Exam (100 points) Honor Code: Each question is worth 10 points. There is one bonus question worth 5 points. In contrast to the homework assignments, you may not collaborate on this final exam. You
Algebras with finite descriptions
Algebras with finite descriptions André Nies The University of Auckland July 19, 2005 Part 1: FA-presentability A countable structure in a finite signature is finite-automaton presentable (or automatic)
Some Applications of MATH 1090 Techniques
1 Some Applications of MATH 1090 Techniques 0.1 A simple problem about Nand gates in relation to And gates and inversion (Not) Gates. In this subsection the notation A means that A is proved from the axioms
Preliminary check: are the terms that we are adding up go to zero or not? If not, proceed! If the terms a n are going to zero, pick another test.
Throughout these templates, let series. be a series. We hope to determine the convergence of this Divergence Test: If lim is not zero or does not exist, then the series diverges. Preliminary check: are
Informal Statement Calculus
FOUNDATIONS OF MATHEMATICS Branches of Logic 1. Theory of Computations (i.e. Recursion Theory). 2. Proof Theory. 3. Model Theory. 4. Set Theory. Informal Statement Calculus STATEMENTS AND CONNECTIVES Example
ULTRAFILTER AND HINDMAN S THEOREM
ULTRAFILTER AND HINDMAN S THEOREM GUANYU ZHOU Abstract. In this paper, we present various results of Ramsey Theory, including Schur s Theorem and Hindman s Theorem. With the focus on the proof of Hindman
Answers to the CSCE 551 Final Exam, April 30, 2008
Answers to the CSCE 55 Final Exam, April 3, 28. (5 points) Use the Pumping Lemma to show that the language L = {x {, } the number of s and s in x differ (in either direction) by at most 28} is not regular.
Euler s method for solving a differential equation (approximately)
Euler s method for solving a differential equation (approximately) Department of Mathematics, UW - Madison March 4, 2013 A chemical reaction A B A B A A A chemical reactor contains two kinds of molecules,
Monotonic Learning, Interactive Realizers and Monads
Monotonic Learning, Interactive Realizers and Monads Ugo de Liguoro Dipartimento di Informatica, Università di Torino joint work with Stefano Berardi TYPES 09, Aussois, May 13th 2009 1 Motivation 2 The
Introduction to Turing Machines
Introduction to Turing Machines Deepak D Souza Department of Computer Science and Automation Indian Institute of Science, Bangalore. 12 November 2015 Outline 1 Turing Machines 2 Formal definitions 3 Computability
Analytic Number Theory Solutions
Analytic Number Theory Solutions Sean Li Cornell University sxl6@cornell.edu Jan. 03 Introduction This document is a work-in-progress solution manual for Tom Apostol s Introduction to Analytic Number Theory.
Opleiding Informatica
Opleiding Informatica Tape-quantifying Turing machines in the arithmetical hierarchy Simon Heijungs Supervisors: H.J. Hoogeboom & R. van Vliet BACHELOR THESIS Leiden Institute of Advanced Computer Science
(9-5) Def Turing machine, Turing computable functions. (9-7) Def Primitive recursive functions, Primitive recursive implies Turing computable.
(9-5) Def Turing machine, Turing computable functions. A.Miller M773 Computability Theory Fall 2001 (9-7) Def Primitive recursive functions, Primitive recursive implies Turing computable. 1. Prove f(n)
Stirling Numbers of the 1st Kind
Daniel Reiss, Colebrook Jackson, Brad Dallas Western Washington University November 28, 2012 Introduction The set of all permutations of a set N is denoted S(N), while the set of all permutations of {1,
dy dx dx = 7 1 x dx dy = 7 1 x dx e u du = 1 C = 0
1. = 6x = 6x = 6 x = 6 x x 2 y = 6 2 + C = 3x2 + C General solution: y = 3x 2 + C 3. = 7 x = 7 1 x = 7 1 x General solution: y = 7 ln x + C. = e.2x = e.2x = e.2x (u =.2x, du =.2) y = e u 1.2 du = 1 e u
Logic and Proof. Aiichiro Nakano
Logic and Proof Aiichiro Nakano Collaboratory for Advanced Computing & Simulations Department of Computer Science Department of Physics & Astronomy Department of Chemical Engineering & Materials Science
COMP 182 Algorithmic Thinking. Proofs. Luay Nakhleh Computer Science Rice University
COMP 182 Algorithmic Thinking Proofs Luay Nakhleh Computer Science Rice University 1 Reading Material Chapter 1, Section 3, 6, 7, 8 Propositional Equivalences The compound propositions p and q are called
Foundations of mathematics. 5. Galois connections
Foundations of mathematics 5. Galois connections Sylvain Poirier http://settheory.net/ The notion of Galois connection was introduced in 2.11 (see http://settheory.net/set2.pdf) with its first properties.
Language Learning Problems in the Principles and Parameters Framework
Language Learning Problems in the Principles and Parameters Framework Partha Niyogi Presented by Chunchuan Lyu March 22, 2016 Partha Niyogi (presented by c.c. lyu) Language Learning March 22, 2016 1 /
MATH FINAL EXAM REVIEW HINTS
MATH 109 - FINAL EXAM REVIEW HINTS Answer: Answer: 1. Cardinality (1) Let a < b be two real numbers and define f : (0, 1) (a, b) by f(t) = (1 t)a + tb. (a) Prove that f is a bijection. (b) Prove that any
Probabilistic Learning of Indexed Families under Monotonicity Constraints
Probabilistic Learning of Indexed Families under Monotonicity Constraints Hierarchy Results and Complexity Aspects vorgelegt von Diplom Mathematikerin Lea Meyer aus Kehl Dissertation zur Erlangung des
MACHINE LEARNING - CS671 - Part 2a The Vapnik-Chervonenkis Dimension
MACHINE LEARNING - CS671 - Part 2a The Vapnik-Chervonenkis Dimension Prof. Dan A. Simovici UMB Prof. Dan A. Simovici (UMB) MACHINE LEARNING - CS671 - Part 2a The Vapnik-Chervonenkis Dimension 1 / 30 The
Lecture Notes. Advanced Discrete Structures COT S
Lecture Notes Advanced Discrete Structures COT 4115.001 S15 2015-01-13 Recap Divisibility Prime Number Theorem Euclid s Lemma Fundamental Theorem of Arithmetic Euclidean Algorithm Basic Notions - Section
Learning Relational Patterns
Learning Relational Patterns Michael Geilke 1 and Sandra Zilles 2 1 Fachbereich Informatik, Technische Universität Kaiserslautern D-67653 Kaiserslautern, Germany geilke.michael@gmail.com 2 Department of
A Theory of Fault Based Testing. Larry J. Morell Presented by: Joonghoon Lee
A Theory of Fault Based Testing Larry J. Morell Presented by: Joonghoon Lee A Reliable Test? A test whose success implies Program Correctness Unattainable in general Quality Measures Desirable to have
Relations to first order logic
An Introduction to Description Logic IV Relations to first order logic Marco Cerami Palacký University in Olomouc Department of Computer Science Olomouc, Czech Republic Olomouc, November 6 th 2014 Marco
Logic Part I: Classical Logic and Its Semantics
Logic Part I: Classical Logic and Its Semantics Max Schäfer Formosan Summer School on Logic, Language, and Computation 2007 July 2, 2007 1 / 51 Principles of Classical Logic classical logic seeks to model
Degrees of Streams. Jörg Endrullis Dimitri Hendriks Jan Willem Klop. Streams Seminar Nijmegen, 20th April Vrije Universiteit Amsterdam
Degrees of Streams Jörg Endrullis Dimitri Hendriks Jan Willem Klop Vrije Universiteit Amsterdam Streams Seminar Nijmegen, 20th April 2010 Complexity of streams Complexity measures for infinite streams:
Krivine s Intuitionistic Proof of Classical Completeness (for countable languages)
Krivine s Intuitionistic Proof of Classical Completeness (for countable languages) Berardi Stefano Valentini Silvio Dip. Informatica Dip. Mat. Pura ed Applicata Univ. Torino Univ. Padova c.so Svizzera
Ma/CS 117c Handout # 5 P vs. NP
Ma/CS 117c Handout # 5 P vs. NP We consider the possible relationships among the classes P, NP, and co-np. First we consider properties of the class of NP-complete problems, as opposed to those which are
Seminaar Abstrakte Wiskunde Seminar in Abstract Mathematics Lecture notes in progress (27 March 2010)
http://math.sun.ac.za/amsc/sam Seminaar Abstrakte Wiskunde Seminar in Abstract Mathematics 2009-2010 Lecture notes in progress (27 March 2010) Contents 2009 Semester I: Elements 5 1. Cartesian product
CSE 20 DISCRETE MATH SPRING
CSE 20 DISCRETE MATH SPRING 2016 http://cseweb.ucsd.edu/classes/sp16/cse20-ac/ Today's learning goals Describe computer representation of sets with bitstrings Define and compute the cardinality of finite
Peano Arithmetic. CSC 438F/2404F Notes (S. Cook) Fall, Goals Now
CSC 438F/2404F Notes (S. Cook) Fall, 2008 Peano Arithmetic Goals Now 1) We will introduce a standard set of axioms for the language L A. The theory generated by these axioms is denoted PA and called Peano
Computational Learning Theory
1 Computational Learning Theory 2 Computational learning theory Introduction Is it possible to identify classes of learning problems that are inherently easy or difficult? Can we characterize the number
Lecture Notes on Data Abstraction
Lecture Notes on Data Abstraction 15-814: Types and Programming Languages Frank Pfenning Lecture 14 October 23, 2018 1 Introduction Since we have moved from the pure λ-calculus to functional programming
Lecture 5: Moment generating functions
Lecture 5: Moment generating functions Definition 2.3.6. The moment generating function (mgf) of a random variable X is { x e tx f M X (t) = E(e tx X (x) if X has a pmf ) = etx f X (x)dx if X has a pdf
Introduction to proofs. Niloufar Shafiei
Introduction to proofs Niloufar Shafiei proofs Proofs are essential in mathematics and computer science. Some applications of proof methods Proving mathematical theorems Designing algorithms and proving
CS 125 Section #10 (Un)decidability and Probability November 1, 2016
CS 125 Section #10 (Un)decidability and Probability November 1, 2016 1 Countability Recall that a set S is countable (either finite or countably infinite) if and only if there exists a surjective mapping
Degree Spectra of Relations on Ordinals
Degree Spectra of Relations on Ordinals Matthew Wright University of Chicago mrwright@math.uchicago.edu November 15, 2012 Background Let A be a structure, and let R A n be an n-ary relation on A. Definition
Advanced Automata Theory 11 Regular Languages and Learning Theory
Advanced Automata Theory 11 Regular Languages and Learning Theory Frank Stephan Department of Computer Science Department of Mathematics National University of Singapore fstephan@comp.nus.edu.sg Advanced
Computational Learning Theory. CS534 - Machine Learning
Computational Learning Theory CS534 Machine Learning Introduction Computational learning theory Provides a theoretical analysis of learning Shows when a learning algorithm can be expected to succeed Shows
Chapter 3. Formal Number Theory
Chapter 3. Formal Number Theory 1. An Axiom System for Peano Arithmetic (S) The language L A of Peano arithmetic has a constant 0, a unary function symbol, a binary function symbol +, binary function symbol,
Notes on Satisfiability-Based Problem Solving. First Order Logic. David Mitchell October 23, 2013
Notes on Satisfiability-Based Problem Solving First Order Logic David Mitchell mitchell@cs.sfu.ca October 23, 2013 Preliminary draft. Please do not distribute. Corrections and suggestions welcome. In this
Bounding Prime Models
Bounding Prime Models Barbara F. Csima, Denis R. Hirschfeldt, Julia F. Knight, and Robert I. Soare Contents 1 Introduction 3 2 The properties: (P0) (P8), and (U0) 6 2.1 Definitions and notations for the
Measures and Measure Spaces
Chapter 2 Measures and Measure Spaces In summarizing the flaws of the Riemann integral we can focus on two main points: 1) Many nice functions are not Riemann integrable. 2) The Riemann integral does not
Computational Learning Theory: Shattering and VC Dimensions. Machine Learning. Spring The slides are mainly from Vivek Srikumar
Computational Learning Theory: Shattering and VC Dimensions Machine Learning Spring 2018 The slides are mainly from Vivek Srikumar 1 This lecture: Computational Learning Theory The Theory of Generalization
Induction on Failing Derivations
Induction on Failing Derivations Technical Report PL-Sep13 September 2013, with addenda from Spring 2016 ay Ligatti Department of Computer Science and Engineering University of South Florida Abstract A
3. The Logic of Quantified Statements Summary. Aaron Tan August 2017
3. The Logic of Quantified Statements Summary Aaron Tan 28 31 August 2017 1 3. The Logic of Quantified Statements 3.1 Predicates and Quantified Statements I Predicate; domain; truth set Universal quantifier,
Computable Functions
Computable Functions Part I: Non Computable Functions Computable and Partially Computable Functions Computable Function Exists a Turing Machine M -- M Halts on All Input -- M(x) = f (x) Partially Computable
55 Separable Extensions
55 Separable Extensions In 54, we established the foundations of Galois theory, but we have no handy criterion for determining whether a given field extension is Galois or not. Even in the quite simple
Lecture Learning infinite hypothesis class via VC-dimension and Rademacher complexity;
CSCI699: Topics in Learning and Game Theory Lecture 2 Lecturer: Ilias Diakonikolas Scribes: Li Han Today we will cover the following 2 topics: 1. Learning infinite hypothesis class via VC-dimension and
CHAPTER 2. FIRST ORDER LOGIC
CHAPTER 2. FIRST ORDER LOGIC 1. Introduction First order logic is a much richer system than sentential logic. Its interpretations include the usual structures of mathematics, and its sentences enable us
From Satisfiability to Linear Algebra
From Satisfiability to Linear Algebra Fangzhen Lin Department of Computer Science Hong Kong University of Science and Technology Clear Water Bay, Kowloon, Hong Kong Technical Report August 2013 1 Introduction
5. Series Solutions of ODEs
Advanced Engineering Mathematics 5. Series solutions of ODEs 1 5. Series Solutions of ODEs 5.1 Power series method and Theory of the power series method Advanced Engineering Mathematics 5. Series solutions
A UNIVERSAL SEQUENCE OF CONTINUOUS FUNCTIONS
A UNIVERSAL SEQUENCE OF CONTINUOUS FUNCTIONS STEVO TODORCEVIC Abstract. We show that for each positive integer k there is a sequence F n : R k R of continuous functions which represents via point-wise
CS 514, Mathematics for Computer Science Mid-semester Exam, Autumn 2017 Department of Computer Science and Engineering IIT Guwahati
CS 514, Mathematics for Computer Science Mid-semester Exam, Autumn 2017 Department of Computer Science and Engineering IIT Guwahati Important 1. No questions about the paper will be entertained during
5. Peano arithmetic and Gödel s incompleteness theorem
5. Peano arithmetic and Gödel s incompleteness theorem In this chapter we give the proof of Gödel s incompleteness theorem, modulo technical details treated in subsequent chapters. The incompleteness theorem
Some Independence Results for Control Structures in Complete Numberings
Some Independence Results for Control Structures in Complete Numberings Sanjay Jain School of Computing National University of Singapore Singapore 119260, Rep. of Singapore sanjay@comp.nus.edu.sg Jochen
Homework 8/Solutions
MTH 309-4 Linear Algebra I F11 Homework 8/Solutions Section Exercises 6.2 1,2,9,12,16,21 Section 6.2 Exercise 2. For each of the following functions, either show the function is onto by choosing an arbitrary
Learning indexed families of recursive languages from positive data: a survey
Learning indexed families of recursive languages from positive data: a survey Steffen Lange a, Thomas Zeugmann b, and Sandra Zilles c,1, a Fachbereich Informatik, Hochschule Darmstadt, Haardtring 100,
Structural Induction
Structural Induction In this lecture we ll extend the applicability of induction to many universes, ones where we can define certain kinds of objects by induction, in addition to proving their properties
Modal logics: an introduction
Modal logics: an introduction Valentin Goranko DTU Informatics October 2010 Outline Non-classical logics in AI. Variety of modal logics. Brief historical remarks. Basic generic modal logic: syntax and
Learnability and Numberings of Families of Computably Enumerable Sets
Learnability and Numberings of Families of Computably Enumerable Sets Klaus Ambos-Spies These notes contain most of the material covered in the course Learnability and Numberings of Families of Computably
Online Learning, Mistake Bounds, Perceptron Algorithm
Online Learning, Mistake Bounds, Perceptron Algorithm 1 Online Learning So far the focus of the course has been on batch learning, where algorithms are presented with a sample of training data, from which
Hence C has VC-dimension d iff. π C (d) = 2 d. (4) C has VC-density l if there exists K R such that, for all
1. Computational Learning Abstract. I discuss the basics of computational learning theory, including concept classes, VC-dimension, and VC-density. Next, I touch on the VC-Theorem, ɛ-nets, and the (p,