So... why do we keep having this debate: rules/symbols vs. prototypes/connections?

Transcription

1 So... wy o we keep avng ts ebate: rules/symbols vs. prototypes/connectons?

2 So... Te real problem: a spurous contest between logc an probablty. Neter logc nor probablty on ts own s suffcent to account for uman cognton: Generatvty Systematcty Recurson an abstracton Flexblty Effectve uner great uncertanty e.g., sparse ata Wat we really nee s to unerstan ow logc an probablty can work togeter.

3 So... Te real problem: a spurous contest between logc an probablty. A confuson between knowlege representatons an nference processes: Graeness or fuzzness oesn t necessarly mean tat te knowlege representatons lack structure or rules -- merely tat te nference processes ncorporate uncertanty. robablstc nference over structure representatons s wat we nee.

4 So... wy o we keep avng ts ebate: rules/symbols vs. prototypes/connectons? wy as t taken Cogntve Scence muc longer to get over t tan AI?

5 Introucton to Bayesan nference

6 Representatveness n reasonng Wc sequence s more lkely to be prouce by flppng a far con? HHTHT HHHHH

7 A reasonng fallacy Kaneman & Tversky: people juge te probablty of an outcome base on te extent to wc t s representatve of te generatng process.

8 Not wre for probablty? Slovc, Fscoff, an Lctensten 1976: It appears tat people lack te correct programs for many mportant jugmental tasks... t may be argue tat we ave not a te opportunty to evolve an ntellect capable of ealng conceptually wt uncertanty. Goul 1992: Our mns are not bult for watever reason to work by te rules of probablty.

9 Arstotle 4t century B.C. In On te eavens, Arstotle asks weter te stars move nepenently or weter tey are all fxe to some spere. He observes tat stars movng n large crcles near te celestal equator take te same tme to rotate as tose near te polestar, wc rotate n small crcles. Infers a common cause: If, on te oter an, te arrangement was a cance combnaton, te concence n every case of a greater crcle wt a swfter movement of te star contane n t s too muc to beleve. In one or two cases, t mgt not nconcevably fall out so, but to magne t n every case alke s a mere fcton. Beses, cance as no place n tat wc s natural, an wat appens everywere an n every case s no matter of cance.

10 Image remove ue to copyrgt conseratons. lease see: Halley. "Motuum Cometarum n Orbe arabolco Elementa Astronomca." In "Astronomae Cometae Synopss." lsopcal Transactons Transcrpt avalable at ttp://

11 Image remove ue to copyrgt conseratons. lease see: Halley. "Motuum Cometarum n Orbe arabolco Elementa Astronomca." In "Astronomae Cometae Synopss." lsopcal Transactons Transcrpt avalable at ttp://

12 A reasonng fallacy Kaneman & Tversky: people juge te probablty of an outcome base on te extent to wc t s representatve of te generatng process. But ow oes representatveness work?

13 rectve versus nuctve reasonng Hypotess H Data D

14 rectve versus nuctve reasonng recton gven H Lkeloo: D H? D

15 rectve versus nuctve reasonng recton gven H Inucton? Lkeloo: D H Representatveness? D gven

16 Bayes rule For ata D an a ypotess H, we ave: H D H D D H osteror probablty : ror probablty : Lkeloo : H D H H D

17 Te orgn of Bayes rule A smple consequence of usng probablty to represent egrees of belef For any two ranom varables: B A B B A A B A B A A B A B A B B A B A B A

18 Wy represent egrees of belef Cox Axoms wt probabltes? necessary to coere wt common sense Dutc Book + Survval of te Fttest f your belefs o not accor wt te laws of probablty, ten you can always be out-gamble by someone wose belefs o so accor. roves a teory of learnng a common currency for combnng pror knowlege an te lessons of experence.

19 Cox Axoms va Jaynes Degrees of belef are represente by real numbers. Qualtatve corresponence wt common sense, e.g.: Bel A f [ Bel A] Consstency: Bel A B g[ Bel A, Bel B A] If a concluson can be reasone n more tan one way, ten every possble way must lea to te same result. All avalable evence soul be taken nto account wen nferrng a egree of belef. Equvalent states of knowlege soul be represente wt equvalent egrees of belef. Acceptng tese axoms mples Bel can be represente as a probablty measure.

20 robablty as propostonal logc wt uncertanty All of probablty teory can be erve from tese two laws plus propostonal logc: A I + A I 1 A B I A, B I A B, I B I Tat s goo: smple, elegant prncples. Tat s ba: ow to work wt structure representatons? More on tat later.

21 Bayesan nference Bayes rule: H D H D D H Wat makes a goo scentfc argument? HD s g f: Hypotess s plausble: H s g Hypotess strongly prects te observe ata: DH s g Data are surprsng: D s low

22 A more useful form of Bayes Ranom varable X enotes a set of mutually exclusve exaustve propostons states of te worl: A useful rule: contonalzaton },, { 1 n x x X K 1 x X j j j y Y y Y x X x X

23 A more useful form of Bayes Ranom varable X enotes a set of mutually exclusve exaustve propostons states of te worl: Bayes rule for more tan two ypoteses: },, { 1 n x x X K 1 x X D H D H D H

24 A more useful form of Bayes Ranom varable X enotes a set of mutually exclusve exaustve propostons states of te worl: Bayes rule for more tan two ypoteses: },, { 1 n x x X K 1 x X H D H H D H D H

25 A more useful form of Bayes Ranom varable X enotes a set of mutually exclusve exaustve propostons states of te worl: Bayes rule for more tan two ypoteses: },, { 1 n x x X K 1 x X

26 Serlock Holmes How often ave I sa to you tat wen you ave elmnate te mpossble watever remans, owever mprobable, must be te trut? Te Sgn of te Four

27 Serlock Holmes How often ave I sa to you tat wen you ave elmnate te mpossble watever remans, owever mprobable, must be te trut? Te Sgn of te Four +

28 Serlock Holmes How often ave I sa to you tat wen you ave elmnate te mpossble watever remans, owever mprobable, must be te trut? Te Sgn of te Four + 0

29 Serlock Holmes How often ave I sa to you tat wen you ave elmnate te mpossble watever remans, owever mprobable, must be te trut? Te Sgn of te Four 1

30 Serlock Holmes How often ave I sa to you tat wen you ave elmnate te mpossble watever remans, owever mprobable, must be te trut? Te Sgn of te Four 1 > 0

31 A reasonng fallacy Kaneman & Tversky: people juge te probablty of an outcome base on te extent to wc t s representatve of te generatng process.

32 Hypoteses n con flppng Descrbe processes by wc D coul be generate D HHTHT Far con, H 0.5 Con wt H θ Markov moel Hen Markov moel... generatve moels

33 Representng generatve moels Grapcal moel notaton earl 1988, Joran 1998 Varables are noes, eges ncate epenency Drecte eges sow causal process of ata generaton HHTHT Far con: H Markov moel: f f +1

34 Moels wt latent structure Not all noes n a grapcal moel nee to be observe Some varables reflect latent structure, use n generatng D but unobserve θ H θ s 1 s 2 s 3 s 4 HHTHT Hen Markov moel: {Far con, Trck con} s

35 Con flppng Comparng two smple ypoteses H 0.5 vs. H 1.0 Comparng smple an complex ypoteses H 0.5 vs. H θ Comparng nfntely many ypoteses H θ : Infer θ

36 Con flppng Comparng two smple ypoteses H 0.5 vs. H 1.0 Comparng smple an complex ypoteses H 0.5 vs. H θ Comparng nfntely many ypoteses H θ : Infer θ

37 Con flppng HHTHT HHHHH Wat process prouce tese sequences?

38 Comparng two smple ypoteses Contrast smple ypoteses: H 1 : far con, H 0.5 H 2 : always eas, H 1.0 Bayes rule: H D H D D H Wt two ypoteses, use os form

39 Bayes rule n os form D: ata H 1, H 2 : H 1 D: H 1 D DH 1 H 1 x H 2 D DH 2 H 2 moels posteror probablty H 1 generate te ata DH 1 : lkeloo of ata uner moel H 1 H 1 : pror probablty H 1 generate te ata

40 Comparng two smple ypoteses H 1 D DH 1 H 1 x H 2 D DH 2 H 2 D: HHTHT H 1, H 2 : far con, always eas DH 1 1/2 5 H 1? DH 2 0 H 2 1-? H 1 D / H 2 D nfnty

41 Comparng two smple ypoteses H 1 D DH 1 H 1 x H 2 D DH 2 H 2 D: HHTHT H 1, H 2 : far con, always eas DH 1 1/2 5 H 1 999/1000 DH 2 0 H 2 1/1000 H 1 D / H 2 D nfnty

42 Comparng two smple ypoteses H 1 D DH 1 H 1 x H 2 D DH 2 H 2 D: HHHHH H 1, H 2 : far con, always eas DH 1 1/2 5 H 1 999/1000 DH 2 1 H 2 1/1000 H 1 D / H 2 D 30

43 Comparng two smple ypoteses H 1 D DH 1 H 1 x H 2 D DH 2 H 2 D: HHHHHHHHHH H 1, H 2 : far con, always eas DH 1 1/2 10 H 1 999/1000 DH 2 1 H 2 1/1000 H 1 D / H 2 D 1

44 Te role of teores Te fact tat HHTHT looks representatve of a far con an HHHHH oes not reflects our mplct teores of ow te worl works. Easy to magne ow a trck all-eas con coul work: g pror probablty. Har to magne ow a trck HHTHT con coul work: low pror probablty.