Representing Knowledge

Transcription

1 Representing Knowledge Dustin Smith MIT Media Lab July 2008 Commonsense Computing MIT MediaLab

2 Big questions How to get it? How to use it? Commonsense Computing MIT MediaLab 2

3 Big questions How to get it? How to use it? Commonsense Computing MIT MediaLab 3

4 Big questions How to get it? How to use it? environment Commonsense Computing MIT MediaLab 4

5 Big questions How to get it? sensory How to use it? environment Commonsense Computing MIT MediaLab 4

6 Big questions How to get it? sensory How to use it? environment knowledge Commonsense Computing MIT MediaLab 4

7 Big questions How to get it? sensory How to use it? environment motor knowledge Commonsense Computing MIT MediaLab 4

8 Math Computer Science / AI Language Commonsense Computing MIT MediaLab 5

9 Representing Items Learning Relations Combining Processes Commonsense Computing MIT MediaLab 6

10 Representing Items Learning Relations Combining Processes Commonsense Computing MIT MediaLab 6

11 Representing Items Math Why math? Compare these two definitions: a) x(y + z) = xy + xz b) Any number that is multiplied by the sum of two numbers is equal to the sum of the products of it and each of the numbers in the first sum. Commonsense Computing MIT MediaLab 7

12 Representing Items Math - a set is a collection of arbitrary items, defined by: (1) listing its members (2) stating a qualifying property (3) stating a set of rules to generate it Examples of sets: A = {1,2,3,4,5} A = {x x is a positive integer less than 6} A = a) 1 A b) if x A and x < 5, then x + 1 A Commonsense Computing MIT MediaLab 8

13 Representing Items Math All of the items referred to by the set are known as its extension. {dog, bear, kangaroo,...} The specification of the set s criteria is its intension: {x mammal(x) brown(x)} Commonsense Computing MIT MediaLab 9

14 Representing Items Math Common set operations: in a {a, b, c} not in d / {a, b, c} empty set subset {a, b, c} {a, b, c} superset {a, b, c} {a} not subset {a, b, c} {a, b} proper subset {a, b, c} {a, b, c, d} not proper subset {a, b, c} {a, b, c} subset {a, b, c} Commonsense Computing MIT MediaLab 10

15 Representing Items Math union A B intersection A B complement A Universe Ω empty set Cross product:{representing, learning, combining} {items, relations, processes} = {{representing, items}, {representing, relations}, {representing, processes}, {learning, items}, {learning, relations},...} Commonsense Computing MIT MediaLab 11

16 Representing Items? Math It s hard just to talk about items, because items are defined by smaller items (features, properties). And sets of items are related via containment. Animals (set) Commonsense Computing MIT MediaLab 12

17 Representing Items? Math It s hard just to talk about items, because items are defined by smaller items (features, properties). And sets of items are related via containment. Animals (set) Pets (set) Commonsense Computing MIT MediaLab 12

18 Representing Items? Math It s hard just to talk about items, because items are defined by smaller items (features, properties). And sets of items are related via containment. Animals (set) Pets (set) Dogs (set) Commonsense Computing MIT MediaLab 12

19 Representing Items? Math It s hard just to talk about items, because items are defined by smaller items (features, properties). And sets of items are related via containment. Animals (set) Pets (set) Dogs (set) Fido (member) Commonsense Computing MIT MediaLab 12

20 Representing Items? Math Animals A(set) Pets P(set)Dogs D (set) Fido f (member) D A A P f A f D f / P D = {x has four legs(x) howls(x) } Commonsense Computing MIT MediaLab 13

21 Representing Items? Math Animals A(set) Pets P(set)Dogs D (set) Fido f (member) D A A P f A f D f / P relationships between sets/categories D = {x has four legs(x) howls(x) } Commonsense Computing MIT MediaLab 13

22 Representing Items? Math Animals A(set) Pets P(set)Dogs D (set) Fido f (member) D A A P f A f D f / P relationships between sets/categories relationships between sets & members D = {x has four legs(x) howls(x) } Commonsense Computing MIT MediaLab 13

23 Representing Items? Math Animals A(set) Pets P(set)Dogs D (set) Fido f (member) D A A P f A f D f / P relationships between sets/categories relationships between sets & members intensional set definition / membership criterion D = {x has four legs(x) howls(x) } Commonsense Computing MIT MediaLab 13

24 Representing Relations Math A relation is a property that holds (or not) between objects. ard bre crd Relation between two sets: A and B: R A B a b c A e d B R = { a, d, b, e, c, d }

25 Representing Relations Math A relation is a property that holds (or not) between objects. ard bre crd Relation between two sets: A and B: R A B a b c A e d B R = { a, d, b, e, c, d }

26 Representing Relations Math A relation is a property that holds (or not) between objects. ard bre crd Relation between two sets: A and B: R A B a b c e d A R = { a, d, b, e, c, d } B {1, 2} = {2, 1} 1, 2 2, 1

27 Representing Relations Math Properties of relations: Reflexive for every x A, x, x R a b c A a b c B

28 Representing Relations Math Properties of relations: Irreflexive for every x A, x, x R a b c A c a b B

29 Representing Relations Math Properties of relations: Symmetric for every x, y A, y, x R a b a b A B

30 Representing Relations Math Properties of relations: Asymmetric for every x, y A, y, x / R a b c A e d B

31 Representing Relations Math Properties of relations: Antisymmetric if whenever x, y and y, x R, then x = y a b c A a e B reflexive anti symmetric

32 Representing Relations Math Properties of relations: Transitive if whenever x, y and y, z R, then x, z R a b c b A B

33 Representing Relations Partial orderings: a binary relationship that is Math for sets a) reflexive b) antisymmetric c) transitive Linear ordering: a partial ordering where x y or y x for every x, y (e,g. for real numbers, is a linear ordering)

34 Representing Relations Math Lattice: 1. set L 2. partial ordering 3., a b a b = greatest lower bound = least upper bound

35 Representing Relations Math Leibniz Universal Characteristic (1679): Primitive Types: 1 = INSECT 2 = ALIVE 3 = MOSTLY-WATER 5 = IS-STRUGGLING-ARTIST 7 = WORKS-IN-E15-320G 11 = STUDIES-AI Composite Types: 462 = DUSTIN (2 x 3 x 7 x 11)

36 Representing Relations Math Leibniz Universal Characteristic (1679): Primitive Types: 1 = INSECT 2 = ALIVE 3 = MOSTLY-WATER 5 = IS-STRUGGLING-ARTIST 7 = WORKS-IN-E15-320G 11 = STUDIES-AI Composite Types: 462 = DUSTIN (2 x 3 x 7 x 11) a b if b % a == 0 a b a b = g.c.d(a,b) = smallest integer divisible. lattice is bounded

37 Representing Relations Math Leibniz Universal Characteristic (1679): Primitive Types: 1 = INSECT 2 = ALIVE 3 = MOSTLY-WATER 5 = IS-STRUGGLING-ARTIST 7 = WORKS-IN-E15-320G 11 = STUDIES-AI Composite Types: 462 = DUSTIN (2 x 3 x 7 x 11)

38 Representing Relations Math Equivalently, we could use a bit string: Primitive Types: 1 = INSECT 2 = ALIVE 3 = MOSTLY-WATER 5 = IS-STRUGGLING-ARTIST 7 = WORKS-IN-E15-320G 11 = STUDIES-AI Composite Types: 462 = DUSTIN (2 x 3 x 7 x 11) Primitive Types: 2 0 = INSECT 2 1 = ALIV E 2 2 = MOST LY W AT ER 2 3 = ART IST 2 4 = E15320G 2 5 = ST UDIESAI = Dustin

39 Representing Relations Math Equivalently, we could use a bit string: Primitive Types: 1 = INSECT 2 = ALIVE 3 = MOSTLY-WATER 5 = IS-STRUGGLING-ARTIST 7 = WORKS-IN-E15-320G 11 = STUDIES-AI Composite Types: 462 = DUSTIN (2 x 3 x 7 x 11) Primitive Types: 2 0 = INSECT 2 1 = ALIV E 2 2 = MOST LY W AT ER 2 3 = ART IST 2 4 = E15320G 2 5 = ST UDIESAI = Dustin a <= b if b[i] = 1 then a[i] =1 a b a b =logical AND = logical OR

40 Representing Relations Math Equivalently, we could use a bit string: Primitive Types: 1 = INSECT 2 = ALIVE 3 = MOSTLY-WATER 5 = IS-STRUGGLING-ARTIST 7 = WORKS-IN-E15-320G 11 = STUDIES-AI Composite Types: 462 = DUSTIN (2 x 3 x 7 x 11) Primitive Types: 2 0 = INSECT 2 1 = ALIV E 2 2 = MOST LY W AT ER 2 3 = ART IST 2 4 = E15320G 2 5 = ST UDIESAI = Dustin

41 Representing Relations Math Equivalently, we could use a bit string: Primitive Types: 1 = INSECT 2 = ALIVE 3 = MOSTLY-WATER 5 = IS-STRUGGLING-ARTIST 7 = WORKS-IN-E15-320G 11 = STUDIES-AI Composite Types: 462 = DUSTIN (2 x 3 x 7 x 11) Primitive Types: 2 0 = INSECT 2 1 = ALIV E 2 2 = MOST LY W AT ER 2 3 = ART IST 2 4 = E15320G 2 5 = ST UDIESAI = Dustin a <= b if b[i] = 1 then a[i] =1 a b a b =logical AND = logical OR

42 Representing Relations Math Equivalently, we could use a bit string: Primitive Types: 1 = INSECT 2 = ALIVE 3 = MOSTLY-WATER 5 = IS-STRUGGLING-ARTIST 7 = WORKS-IN-E15-320G 11 = STUDIES-AI Composite Types: 462 = DUSTIN (2 x 3 x 7 x 11) Primitive Types: 2 0 = INSECT 2 1 = ALIV E 2 2 = MOST LY W AT ER - These markers only a <= b if b[i] = 1 then a[i] =1 a b a b 2 3 = ART IST 2 4 = E15320G 2 5 = ST UDIESAI represent conjunctions! (ands) = Dustin =logical AND = logical OR

43 Representing Relations People tried this! - Masterman (1961) - defined 15,000 words in terms of 1000 primitives - Schank (1975) - reduced the number of primitive acts to 11

44 Representing Relations Math People tried this! - Masterman (1961) - defined 15,000 words in terms of 1000 primitives - Schank (1975) - reduced the number of primitive acts to 11 But,,, - No linguistic/psychological evidence for universal set of primitives - Languages have families of synonyms (glad, happy, cheerful) with slightly different meanings -- not disjoint either-or groups.

45 Representing Items Math Propositional Logic Sentences are statements that have a truth value S {true, false} P = it will rain today Q = dustin will wear an umbrella P Q?

46 Representing Items Math Logic: Sentential Connectives Connective Symbol English Not P not P And P Q P and Q Or P Q P or Q Implies P Q If P then Q Q if P Q is a necessary condition of P P is a sufficient condition of Q Biconditional P Q P is a sufficient and necessary condition of Q Q is a sufficient and necessary condition of P

47 Representing Items Math Logic: Inference Inference is driving new knowledge from old knowledge. Deduction, is a set of rules for truth-preserving transformations over logical statements.

48 Representing Items Math Propositional Logic, rules of inference Valid proposition: If elephants have wings then, 2+2 = 5

49 Representing Items CS/AI Some representational ideas are in the semantics of the programming language (OOP, Von Neumann, functional), and the programmer can also extend them. Abstraction allows us to name increasingly complicated procedures and data types. Hiding the implementation complexity behind a simple name -- separating the representation from the function. feed(*food) pet(*instrument) Dog object bark() pee() name weight { hidden from you } Commonsense Computing MIT MediaLab 35

50 Representing Items CS/AI Inheritance: Commonsense Computing MIT MediaLab 36

51 Representing Inheritance: Items CS/AI class Animal gender = FMALE; feed(*food) pet(*instrument) name weight gender Dog object bark() pee() Animal { hidden from you } end class Dog < Animal name, weight =,0 bark(); pee(); end Commonsense Computing MIT MediaLab 37

52 Representing Items Language Three main views of category representations: 1. Sufficient and necessary conditions / logic. (categories like game have no common properties) 2. Exemplars - all instances stored 3. Prototypes - one best representative Commonsense Computing MIT MediaLab 38

53 Representing Items Language Exemplars Prototypes Commonsense Computing MIT MediaLab 39

54 Representing Items Language In language items are words, objects are like nouns. Their meaning is contextdependent. Words have various semantic traits when interacting with other words. Commonsense Computing MIT MediaLab 40

55 Representing Items Language Semantic trait: degree and mode of participation: 1. criterial 2. expected 3. possible 4. unexpected 5. excluded textual entailment It s a dog - It s an animal It s a dog - It s a fish Commonsense Computing MIT MediaLab 41

56 Representing Items Language Semantic trait: degree and mode of participation: 1. criterial 2. expected 3. possible 4. unexpected 5. excluded It s a dog, but it can bark. It s a dog, but it can t bark. expected/unexpected traits It s a dog, but it can sing. It s a dog, but it can t sing. Commonsense Computing MIT MediaLab 42

57 Representing Items Language Semantic trait: degree and mode of participation: 1. criterial 2. expected 3. possible 4. unexpected 5. excluded It s a dog, but it can bark. It s a dog, but it can t bark. expected/unexpected traits It s a dog, but it can sing. It s a dog, but it can t sing. Expressive paradoxes Commonsense Computing MIT MediaLab 43

58 Representing Items Language Semantic trait: degree and mode of participation: 1. criterial 2. expected 3. possible 4. unexpected 5. excluded if both 2+3 are expressive paradox, then it s possible It s a dog and it s brown (normal) It s a dog, but it s brown (why shouldn t it be?) Commonsense Computing MIT MediaLab 44

59 Representing Items Learning Relations Combining Processes Commonsense Computing MIT MediaLab 45

60 Representing Items Learning Relations Combining Processes Commonsense Computing MIT MediaLab 45

61 Learning Items Philo. - Deduction: Derive new knowledge by exploiting the structure of old knowledge All Splash Students are Smart John is a Splash Student

62 Learning Items Philo. - Deduction: Derive new knowledge by exploiting the structure of old knowledge All Splash Students are Smart John is a Splash Student John is Smart

63 Learning Items Philo. - Deduction: Derive new knowledge by exploiting the structure of old knowledge All Splash Students are Smart John is a Splash Student John is Smart - by applying inference meta-rules (modus ponens)

64 Learning Items Philo. - Deduction: Derive new knowledge by exploiting the structure of old knowledge - Induction: Learning generalizations to fit data John is Smart, in Splash and 10 years old. Lisa is Smart, in Splash and 10 years old. Joe is Smart, in Splash and 10 years old.

65 Learning Items Philo. - Deduction: Derive new knowledge by exploiting the structure of old knowledge - Induction: Learning generalizations to fit data John is Smart, in Splash and 10 years old. Lisa is Smart, in Splash and 10 years old. Joe is Smart, in Splash and 10 years old. People in Spash are Smart

66 Learning Items Philo. - Deduction: Derive new knowledge by exploiting the structure of old knowledge - Induction: Learning generalizations to fit data John is Smart, in Splash and 10 years old. Lisa is Smart, in Splash and 10 years old. Joe is Smart, in Splash and 10 years old. People named John, Lisa or Joe are smart?

67 Learning Items Philo. - Deduction: Derive new knowledge by exploiting the structure of old knowledge - Induction: Learning generalizations to fit data John is Smart, in Splash and 10 years old. Lisa is Smart, in Splash and 10 years old. Joe is Smart, in Splash and 10 years old. People 10 years old are smart

68 Learning Items Philo. Problems of induction - Induction is never justified. Hume s problem: will the sun rise tomorrow? Assume the past is like the future? - Inductions are always biased. A priori, all hypotheses are equally likely? - Accidental versus law-like hypotheses. Which properties can be generalized to larger classes?

69 Learning Items Philo. Concept Learning - Functionally, a concept is a mental representation that divides the world into positive and negative classes. chair f(x) {true, false}

70 Learning Items Philo. Concept Learning - Functionally, a concept is a mental representation that divides the world into positive and negative classes. chair f(x) {true, false}

71 slides from Josh Tenenbaum s 9.66 Bruner, Goodnow, Austin (1956)

72 Describing the Microworld - Shapes = {,, } - Number = {1, 2, 3} - Texture = {Shaded, Light, Dark} - Frame = {Single, Double, Triple} Number of Concepts: Bruner, Goodnow, Austin (1956)

73 Describing the Microworld - Shapes = {,, } - Number = {1, 2, 3} - Texture = {Shaded, Light, Dark} - Frame = {Single, Double, Triple} Number of Concepts: = Shapes Number T exture frame Bruner, Goodnow, Austin (1956)

74 Describing the Microworld - Shapes = {,, } - Number = {1, 2, 3} - Texture = {Shaded, Light, Dark} - Frame = {Single, Double, Triple} Number of Concepts: = Shapes Number T exture frame = = 81 Bruner, Goodnow, Austin (1956)

75 Describing the Microworld - Shapes = {,, } - Number = {1, 2, 3} - Texture = {Shaded, Light, Dark} - Frame = {Single, Double, Triple} 9 Number of Concepts: = Shapes Number T exture frame = = 81 9 Bruner, Goodnow, Austin (1956)

76 slides from Josh Tenenbaum s 9.66

77 + slides from Josh Tenenbaum s 9.66

78 + slides from Josh Tenenbaum s 9.66

79 + slides from Josh Tenenbaum s 9.66

80 + slides from Josh Tenenbaum s 9.66

81 + + slides from Josh Tenenbaum s 9.66

82 + + slides from Josh Tenenbaum s 9.66

83 + + slides from Josh Tenenbaum s 9.66

84 + + slides from Josh Tenenbaum s

85 + striped and three borders + slides from Josh Tenenbaum s

86 Describing the Microworld - Shapes = {,, } - Number = {1, 2, 3} - Texture = {Shaded, Light, Dark} - Frame = {Single, Double, Triple} Number of percepts: Bruner, Goodnow, Austin (1956)

87 Describing the Microworld - Shapes = {,, } - Number = {1, 2, 3} - Texture = {Shaded, Light, Dark} - Frame = {Single, Double, Triple} Number of percepts: = Shapes Number T exture frame Bruner, Goodnow, Austin (1956)

88 Describing the Microworld - Shapes = {,, } - Number = {1, 2, 3} - Texture = {Shaded, Light, Dark} - Frame = {Single, Double, Triple} Number of percepts: = Shapes Number T exture frame = = 81 Bruner, Goodnow, Austin (1956)

89 Describing the Microworld - Shapes = {,, } - Number = {1, 2, 3} - Texture = {Shaded, Light, Dark} - Frame = {Single, Double, Triple} 9 Number of percepts: = Shapes Number T exture frame = = 81 9 Bruner, Goodnow, Austin (1956)

90

91

92

93 generalization lattice

94

95 Occam s Razor - Favor the simple hypotheses when multiple ones fit the data = = = f(x) θ 1 x + θ 2 f(x) θ 1 x 2 + θ 2 x + θ 3 f(x) θ 1 x 7 + θ 2 x 6 + θ 3 x 5 + θ 4 x 4 + θ 5 x 3 + θ 6 x 2 + θ 7 x + θ 8

96 Occam s Razor - Favor the simple hypotheses when multiple ones fit the data = = = f(x) θ 1 x + θ 2 f(x) θ 1 x 2 + θ 2 x + θ 3 f(x) θ 1 x 7 + θ 2 x 6 + θ 3 x 5 + θ 4 x 4 + θ 5 x 3 + θ 6 x 2 + θ 7 x + θ 8

97 Occam s Razor - Favor the simple hypotheses when multiple ones fit the data = = = f(x) θ 1 x + θ 2 f(x) θ 1 x 2 + θ 2 x + θ 3 f(x) θ 1 x 7 + θ 2 x 6 + θ 3 x 5 + θ 4 x 4 + θ 5 x 3 + θ 6 x 2 + θ 7 x + θ 8

98 Occam s Razor - Favor the simple hypotheses when multiple ones fit the data = = = f(x) θ 1 x + θ 2 f(x) θ 1 x 2 + θ 2 x + θ 3 f(x) θ 1 x 7 + θ 2 x 6 + θ 3 x 5 + θ 4 x 4 + θ 5 x 3 + θ 6 x 2 + θ 7 x + θ 8

99 Occam s Razor - Favor the simple hypotheses when multiple ones fit the data = = = f(x) θ 1 x + θ 2 f(x) θ 1 x 2 + θ 2 x + θ 3 f(x) θ 1 x 7 + θ 2 x 6 + θ 3 x 5 + θ 4 x 4 + θ 5 x 3 + θ 6 x 2 + θ 7 x + θ 8

100 Occam s Razor - Favor the simple hypotheses when multiple ones fit the data = = = f(x) θ 1 x + θ 2 f(x) θ 1 x 2 + θ 2 x + θ 3 f(x) θ 1 x 7 + θ 2 x 6 + θ 3 x 5 + θ 4 x 4 + θ 5 x 3 + θ 6 x 2 + θ 7 x + θ 8

101 What is the function between x and y? - Given examples of x, y pairs, learn function f(x) y slides from Josh Tenenbaum s 9.66

102 What is the function between x and y? - Given examples of x, y pairs, learn function f(x) y slides from Josh Tenenbaum s 9.66

103 What is the function between x and y? - Given examples of x, y pairs, learn function f(x) y f(x) = sin(x) slides from Josh Tenenbaum s 9.66