Computational Learning Theory Extending Patterns with Deductive Inference

Transcription

1 Computational Learning Theory Extending Patterns with Deductive Inference Akihiro Yamamoto 山本章博 1

2 Contents What about a pair of patterns? Elementary formal systems Transferring FA into EFS Transferring CFG into EFS 2

3 What about a pair of patterns? 3

4 Outputting a Pair of Patterns C = {babaabb aaab baaab aabb abab baaaaab abbb aaabb baaab} abaaab aabb abab abbb aaabb L(axb) baaab baaaaab babaabb L(bxyzb) L(axyb) abaaab aaabb aabb abab abbb L(xyzb) L(bxyzb) babba baaab baaaaab babaabb ababa 4

5 Outputting a Pair of Patterns C = {babaabb aaab baaab aabb abab baaaaab abbb aaabb baaab} abaaab aabb abab abbb aaabb L(axyb) baaab baaaaab aaabb babaabb L(xaayb) L(xyzb) L(axyb) abaaab aaabb aabb abab abbb L(xaayb) baaab babba baaaaab babaabb ababa 5

6 Downward Coverset Algorithm Given a data set C For each minimally common anti-unifier of C For each a pair 1, 2 of patterns just beneth in the Hasse Diagram if L( ) C L( 1 ) then make minimally common anti-unifier 1 of C L( 1 ) and minimally common anti-unifier 2 of C L( 1 ) 6

7 Elementary Formal Systems 7

8 Introducing Predicate Symbols Introducing a new class of symbols called predicate symbols. Every predicate symbol is interpreted as a language (a set of strings) or a set of tuples (s 1, s 2,,s n ) of strings. In this course we use symbols p, q, r, which are respectively interpreted as sets P, Q, R,. An atomic formula is a formula of the form p( 1, 2,, n ) where 1, 2,, n are patterns. If n=1 and 1 = s is ground, p(s) is interpreted as s P. Example Some examples of atomic formulae are p(axb), q(ax, by), q(x, bxb), p(aabb), q(aa, bb). The last two formulae are ground. 8

9 Definite Clause (Rules) and EFS An definite clause is a formula of the form p( 1,, n ) q 1 ( 11, ),q 2 ( 21, ),, q k ( k1, ) where 1, 2,, 11,, k1, are patterns. The definite clause is interpreted as for any substitution, if ( 11, ) Q 1, ( 21, ) Q 2,, q k ( k1, ) Q k then ( 1, 2,, n ) P A clause p( 1,, n ) which has no conditions is sometimes called a unit clause. A finite set of definite clause is called an elementary formal system (EFS). [Smullyan 61] 9

10 Examples Some examples of definite clauses are p(ax) r(x) r(b) p(axby) r(x), r(y) q(ax, by) q(x, y) 10

11 11

12 Inference Rules for Definite Clauses A goal clause is a sequence (conjunction) of atomic formulae p 1 ( 11, ),, p k ( k1, ) We use the following two rules Instantiation p( 1, ) q 1 ( 11, ),q 2 ( 21, ),, q k ( k1, ) (p( 1, ) q 1 ( 11, ),q 2 ( 21, ),, q k ( k1, )) Modus Pones p 1 ( 11, ),, p k ( k1, ) p 1 ( 1, ) q 1 ( 11, ), q 1 ( 11, ),, p 2 ( 21, ),, p k ( k1, ) A proof is a continuous application of the inference rules. 12

13 Example of Proof (1) p(axb) p(x) p(aaabbb) p(aaabbb) p(aabb) p(axb) p(x) p(aabb) p(aabb) p(ab) p(ab) p(ab) p(aaabbb) p(axb) p(x) p(aabb) p(axb) p(x) p(ab) p(ab) 13

14 Defining a language by proofs A ground atomic formula p(s 1,,s n ) is provable from an EFS S if there is a proof which derives an empty clause from is p(s 1,,s n ) and S. We define a language with a proof from an EFS. P = L(p, S) ={ s p(s) is provable from S} Example S : p(axb) p(x) p(ab) P = L(p, S) ={ab, aabb, aaabbb, aaabbb, } 14

15 Example of Proof (2) S: p(ax) q(x) q(bx) p(x) p(a) q(b) L(p, S) ={a, ab, aba, abab, } L(q, S) ={b, ba, bab, baba, } p(ababa) q(baba) p(aba) q(ba) p(a) p(ax) q(x) q(bx) p(x) p(ax) q(x) q(bx) p(x) p(a) 15

16 Example of Proof (3) S: p(axb) p(x) p(xy) p(x), p(y) p(ab) L(p, S) ={ab, aabb, aaabbb, abaabb, } p(abaabb) p(xy) p(x), p(y) p(ab), p(aabb) p(ab) p(aabb) p(ab) p(axb) p(x) p(ab) 16

17 States of Languages in FA a a b a b a a a b B B q 0 (abab) q 0 (ax) q 1 (x) q 0 a q 1 a q 1 (bab) b a b q 3 q 2 b a,b 17

18 States of Languages in FA a a b a b a a a b B B q 0 (abab) q 0 (ax) q 1 (x) q 0 a q 1 a q 1 (bab) q 1 (bx) q 2 (x) b a b q 2 (ab) q 3 q 2 b a,b 18

19 States of Languages in FA a a b a b a a a b B B q 0 (abab) q 0 (ax) q 1 (x) q 0 a q 1 a q 1 (bab) q 1 (bx) q 2 (x) b a b q 2 (ab) q 2 (ax) q 1 (x) q 3 a,b q 2 b q 1 (b) 19

20 States of Languages in FA a a b a b a a a b B B q 0 (abab) q 0 (ax) q 1 (x) q 0 a q 1 a q 1 (bab) q 1 (bx) q 2 (x) b a b q 2 (ab) q 2 (ax) q 1 (x) q 3 a,b q 2 b q 1 (b) q 1 (b) 20

21 Representation of Finite State Automata Mathematically, a finite state automaton is represented in the form M=(, S,, s 0, F) where is the alphabet, S is a set of states, : S S is a transition function represented as a transition table, s 0 S is an initial state, F S is a set of final states. q 0 q m F a 1 a n 21

22 Transformation from Finite State Automata We use the elements in S as a predicate symbols. If (p, c) = q, we prepare a definite clause p(cx) q(x) If (p, c) = q and q F we prepare a definite clause p(c) F a 1 a n q 0 q m 22

23 Context Free Grammar A context free grammar is defined as G = (N,, P, S) where N is a finite set of non-terminal symbols (nonterminals, syntactic categories). is a finite set of characters or terminals. P is a set of rules (productions). S N is an initial state. A production is of the form A where A is a non-terminal and is a sequence of terminals and non-terminals. Note: In general definition can be an empty sequence In this course we do not allow to be 23

24 Derivations An application of a production rule A P is written as A where (N ) * This means that only one occurrence of A in a string is replaced with S S + S V S + S S S + S S V + S S S + S V V + V A derivation from to is a finite sequence of applications starting with and ending with : n We write if or there is a derivation from to. 24

25 Context Free Languages The language defined a context free grammar G = (N,, P, S) is L(G) = { w * there is a derivation S w }. A language L is context free if there is a context free grammar G such that L = L(G). Example For G = ({S}, {a b }, {S ab S asb}, S), L(G) = { a n b n n. 25

26 Example 1 a b L = { a n b n n = { a ab b n n times n times ab aabb aaabbb aaaabbbb The language is defined with a set of productions: S ab S asb Some examples of derivations: S ab S asb aabb S asb aasbb aaabbb S asb aasbb aaasbbb aaaabbbb It is easy to show that there is no FA which accepts L. 26

27 Example 2 Mathematical formulae consisting of x, y, +, *, (, and ) correct: x + y, y * ( x + y ), (( x * x ) + x ), x, incorrect: x +, ( y * x + y, ( ), Note: We assume strict use of ( and ). correct: ( x + y ), ( x ) incorrect: x * x * x, y * x + x The set are defined by a set of rules S V V x S S + S V y S S * S S S 27

28 Example 2 (cont.) An example of a derivation: S V S S S V x S S + S S S V y S S S S S S S + S V S + S V V + V y V + V y x + V y x + y 28

29 Transforming CFG to EFS For every production rule P w 1 Q 1 w 2 Q 2 Q n w n+1 P,Q w i we define a definite clause p(w 1 x 1 w 2 x 2 x n w n+1 ) q 1 x 1 q 2 x 2 q n x n Example P apb P apb p(axb) p(x) p(ab) 29

30 Derivation and Proof Productions and derivation P ab P apb P apb aapbb aaabbb EFS and proof p(aaabbb) p(axb) p(x) p(aabb) p(ab) p(axb) p(x) p(ab) 30

31 Notes There is a class of EFS which corresponds to the context sensitive grammar. There is a class of EFS which corresponds to TM. It is not easy to check which class L(p, S) belongs to p(xx) p(x) p(a) 31

32 References Arimura, H., Shinohara, T., and Otsuki, S. : Finding Minimal Generalization for Unions of Pattern Languages and Its application to Inductive Inference from Positive Data, Proc. of STACS 94 (LNCS 775), , Arikawa, S. Shinohara, T., and Yamamoto, A. : Learning Elementary Formal Systems, Theoretical Computer Science, 95(1), ,