Bayesian Decision Theory
|
|
- Luke Townsend
- 6 years ago
- Views:
Transcription
1 Bayesan Decson heory Berln hen 2005 References:. E. Alpaydn Introducton to Machne Learnng hapter 3 2. om M. Mtchell Machne Learnng hapter 6
2 Revew: Basc Formulas for robabltes roduct Rule: probablty A B of a conuncton of two events A and B ( A B ) ( A B ) ( A B ) ( B ) ( B A) ( A) Sum Rule: probablty of a dsuncton of two events A and B ( A B ) ( A) + ( B ) ( A B ) A B heorem of total probablty: f events A A n are mutually exclusve and exhaustve ( A A 0 and A ) ( ) n ( ) n ( B ) ( B A ) n ( B A ) ( A ) A B A 2 A A n MLDM-Berln hen 2
3 Revew: Basc Formulas for robabltes (cont.) han Rule: probablty of a conuncton of many events A A2 A K ( A A K A ) 2 n n ( A ) ( A A ) ( A A A )... ( A A A K A ) n 2 n MLDM-Berln hen 3
4 lassfcaton Illustratve ase : redt Scorng x x x2 hght 2 Low Gven a new applcaton 0 t ncome savngs - rs - rs x ( x) f > hoose 2 otherwse or equvalent ly f > hoose 2 otherwse t [ ] x x ( x) ( x) 2 x 2 ( x) + ( x) Note that 2 MLDM-Berln hen 4
5 lassfcaton (cont.) Bayes lassfer We can use the probablty theory to mae nference from data ( x ) ( x ) ( ) ( x ) x x x : observed data (varable) : class hypothess ( x) : pror probablty of : pror probablty of x : probablty of gven MLDM-Berln hen 5
6 lassfcaton (cont.) alculate the posteror probablty of the concept after havng the observaton x or 2 ombne the pror and what the data tells usng Bayes rule ( x ) posteror ( x ) ( ) ( x ) lelhood pror evdence x 2 and 2 are mutually exclusve and exhaustve classes (concepts) ( x ) ( x ) + ( x ) ( x ) ( ) + ( x ) ( ) MLDM-Berln hen 6
7 MLDM-Berln hen 7 lassfcaton (cont.) Bayes lassfer: Extended to mutually exclusve and exhaustve classes K K K and 0 x x x x x
8 lassfcaton (cont.) Maxmum lelhood classfer he posteror probablty that the data belongs to class L ( x ) ( x ) Have the same classfcaton result as that of Bayes lassfer f the pror probablty ( ) s assumed to be equal to each other max L max ( x) max( x ) max( x)? ( x ) ( ) ( x) MLDM-Berln hen 8
9 lassfcaton: Illustratve ase 2 Does a patent have cancer or not? A patent taes a lab test and result would be x "+" or. If the result comes bac postve ( x "+" ) 2 x " " 2. And we also new that the test returns a correct postve result (+) n only 98% of the cases n whch the dsease s actually present ( ( + ) 0.98) and a correct negatve result (-) n only 97% of the cases n whch the dsease s not present Furthermore of the entre populaton have ths cancer ( 0.008) MLDM-Berln hen 9
10 lassfcaton: Illustratve ase 2 (cont.) Bayes lassfer: ( + ) ( + ) 2 ( + ) ( + ) ( ) + + ) ( ) ( + 2 ) ( 2 ) ( + ) ( ) + + ) ( ) ( + ) 0. 2 ( + ) MLDM-Berln hen 0
11 lassfcaton: Illustratve ase 2 (cont.) Maxmum lelhood classfer: ( + ) 0.98 ( + ) MLDM-Berln hen
12 Losses and Rss Decsons are not always prefect E.g. Loan Applcaton he loss for a hgh-rs applcant erroneously accepted (false acceptance) may be dfferent from that for an erroneously reected low-rs applcant (false reecton) Much crtcal for other cases such as medcal dagnoss or earthquae predcton MLDM-Berln hen 2
13 Expected Rs α Hypothesze that the example x belongs to class Suppose the example actually belongs to some class Def: the Expected Rs for tang acton R λ K ( α x ) λ ( x ) 0 f f A zero-one loss functon All correct decsons have no loss and all error are equally costly α hoose the acton wth mnmum rs α arg mn ( x ) R α MLDM-Berln hen 3
14 Expected Rs (cont.) α hoosng the acton wth mnmum rs s equvalent to choosng the class wth the hghest posteror probablty R K ( α x ) λ ( x ) K - ( x ) ( x ) hoose the acton α wth α arg max ( x ) MLDM-Berln hen 4
15 Expected Rs: Reect Acton Involved Manual Decsons? Wrong decsons (msclassfcaton) may have very hgh cost Resort to a manual decson when automatc system has low certanty of ts decson Defne an addtonal acton of reect (or doubt) 0 f x λ λ f K + otherwse α K + λ 0 λ s the loss ncurred for choosng the (K+)st acton of reect.. K K + MLDM-Berln hen 5
16 MLDM-Berln hen 6 Expected Rs: Reect Acton Involved (cont.) he rs for choosng the reect ((K+)st) acton Recall that the rs for choosng acton λ λ λ α + K K K R x x x K α α K + x x x x K K R - λ α α K + x.. + K K K + α α
17 Expected Rs: Reect Acton Involved (cont.) he optmal decson rule s to: hoose Reect f R hat s hoose f R and ( α x) < R( α x) R ( α x) < R( α x) for all K ( α x) < R( α x) for all K f K + and Reect otherwese λ 0 λ When always reect the chosen acton When always accept the chosen acton ( x) > ( x) ( x) > λ K + for all K MLDM-Berln hen 7
18 Dscrmnant Functons lassfcaton can be thought of as a set of dscrmnant functons g x for each class such that hoose f...k g ( x) max g ( x) g x can be expressed by usng the Bayes s classfer (wth mnmum rs and no addtonal acton of reect) g ( x) R( α x) If the zero-one loss functon s mposed g x can also be expressed by ( x) ( x) g Wth the same ranng result we can have g ( x ) ( x ) ( ) MLDM-Berln hen 8
19 Dscrmnant Functons (cont.) he nstance space thus can be dvded nto K decson regons R... where R R K x g x ( x) max g MLDM-Berln hen 9
20 Dscrmnant Functons (cont.) For two-class problems we can merely defne a sngle dscrmnant functon g ( x) - g ( x) g x 2 ( x) f g > 0 hoose 2 otherwse MLDM-Berln hen 20
21 hoosng Hypotheses* : MA rteron In machne learnng we are nterested n fndng the best (most probable) hypothess (classfer) h c from some hypothess space H gven the observed t t t tranng data set X x r r c t 2 L n { } c h MA arg max h c H ( h X ) c c arg max h c H ( X h ) ( h ) c c ( X ) c c arg max h c H ( X h ) ( h ) c c c A Maxmum a osteror (MA) hypothess h MA MLDM-Berln hen 2
22 hoosng Hypotheses*: ML rteron If we further assume that every hypothess s equally probable a pror e.g. ( h ). he above equaton c h c can be smplfed as: h ML arg max h c H ( X h ) c c A Maxmum Lelhood (ML) hypothess h ML X c h often called the lelhood of the data set c gven h c X c MLDM-Berln hen 22
23 Naïve Bayes lassfer A smplfed approach to the Bayes s classfer x x... he attrbutes of an nstance/example are assumed to be ndependent condtoned on a gven class hypothess Naïve Bayes assumpton: Naïve Bayes lassfer: 2 x d x d ( x ) ( x x x ) ( x ) MA arg arg arg arg 2... max max max max d ( x ) n ( x ) ( ) ( x ) ( x... x ) ( ) d ( ) ( x ) n d n n arg max ( x ) ( ) MLDM-Berln hen 23
24 Naïve Bayes lassfer (cont.) x ( x x x ) A B Illustratve case Gven a data set wth 3-dmensonal Boolean examples x ( x A xb x ) tran a naïve Bayes classfer to predct the classfcaton Attrbute A F F F F Attrbute B F F F F Attrbute F F F lassfcaton D F F F ( D ) / 2 ( D F ) / 2 ( A D ) / 3 ( A F D ) ( B D ) / 3 ( B F D ) ( D ) / 3 ( B F D ) 2 / 3 2 / 3 2 / 3 ( A D F ) / 3 ( A F D F ) 2 / 3 ( B D F ) / 3 ( B F D F ) 2 / 3 ( D F ) 2 / 3 ( B F D F ) / 3 What s the predcted probablty ( D A B F )? What s the predcted probablty? ( D B ) MLDM-Berln hen 24
25 MLDM-Berln hen 25 Naïve Bayes lassfer (cont.) Illustratve case (cont.) () F D F D F B A D D F B A D D F B A F B A D D F B A F B A D ) ( F D F D B D D B D D B B D D B B D
26 How to ran a Naïve Bayes lassfer Naïve_Bayes_Learn(examples) For each target value v maxmum lelhood (ML) estmate of ( v ) ˆ v For each attrbute value a ( v ) maxmum lelhood (ML) estmate of ( a v ) ˆ a x lassfy_new_instance(x) x a a of each attrbute a x v v n or v v ( a ) a x v x( a ) x v x v v a v NB arg max v V ( v ) ( a v ) a x MLDM-Berln hen 26
27 Naïve Bayes: Example 2 onsder layenns agan and new nstance <Outloosunny emperaturecool Humdtyhgh Wndstrong> Want to compute v NB arg v V max ( v ) Outloo sunny v emperature cool v { yes no} ( Humdty hghv ) ( Wnd Strong v ) ( yes) ( Outloo sunny yes) ( emperature cool yes) ( Humdty hgh yes) ( Wnd Strong yes) ( no) ( Outloo sunny no) ( emperature cool no) ( Humdty hgh no) ( Wnd Strong no) v NB no MLDM-Berln hen 27
28 Dealng wth Data Sparseness What f none of the tranng nstances wth target value v have attrbute value a? hen ˆ v ( a v ) NB arg 0 max v V and ( v ) ˆ ( a v ) ypcal soluton s Bayesan estmate for n ˆ... s number of tranng examples for whch n c s number of tranng examples for whch v v and p s pror estmate for ˆ ( a v ) s weght gven to pror (.e. number of vrtual examples) m ˆ ( a v ) n c n + + mp m ˆ Smoothng a v v v a a MLDM-Berln hen 28
29 Example: Learnng to lassfy ext For nstance Learn whch news artcles are of nterest Learn to classfy web pages by topc Naïve Bayes s among the most effectve algorthms What attrbutes shall we use to represent text documents he word occurs n each document poston MLDM-Berln hen 29
30 Example: Learnng to lassfy ext (cont.) arget oncept: Interestng? Document {+-}. Represent each document by vector of words one attrbute per word poston n document 2. Learnng Use tranng examples to estmate (+) (-) doc (doc +) doc w (doc -) Naïve Bayes condtonal ndependence assumpton length ( doc ) doc v ( a w v ) Where a s probablty that word n poston s w w v gven v me Invarant One more assumpton: ( a w v ) ( a m w v ) m a w MLDM-Berln hen 30
31 Example: Learnng to lassfy ext (cont.) Learn_Naïve_Bayes_ext(Examples V). ollect all words and other toens that occur n Examples Vocabulary all dstnct words and other toens n Examples 2. alculate the requred v and w v probablty terms docs subset of Examples for whch the target value s v docs Examples ext a sngle document created by concatenatng all members of docs n total number of words n ext (countng duplcate words multple tmes) w For each word n Vocabulary n number of tmes word w occurs n ( w v ) n + Smoothed ungram n + Vocabulary v MLDM-Berln hen 3
32 Example: Learnng to lassfy ext (cont.) lassfy_naïve_bayes_ext(doc) postons all word postons n Doc that contan toens found n Vocabulary v NB Return where v NB arg max v V ( v ) ( a v ) postons MLDM-Berln hen 32
33 Bayesan Networs remse Naïve Bayes assumpton of condtonal ndependence too restrctve But t s ntractable wthout some such assumptons Bayesan networs descrbe condtonal ndependence among subsets of varables Allows combnng pror nowledge about (n)dependences among varables wth observed tranng data Bayesan Networs also called Bayesan Belef Networs Bayes Nets Belef Networs robablstc Networs Graphcal Models etc. MLDM-Berln hen 33
34 Bayesan Networs (cont.) A smple graphcal notaton for condtonal ndependence assertons and hence for compact specfcaton of full ont dstrbutons Syntax A set of nodes one per varable (dscrete or contnuous) For dscrete varable they can be ether bnary or not A drected acyclc graph (ln/arrow drectly nfluences ) A condtonal dstrbuton for each node gven ts parents ( X arents) X x x In the smplest case condtonal dstrbuton represented as a ondtonal robablty able () gvng the dstrbuton over ( X ) for each combnaton of parent values MLDM-Berln hen 34
35 Bayesan Networs (cont.) E.g. nodes of dscrete bnary varables ondtonal robablty able () S B () F F F F Each node s asserted to be condtonally ndependent of ts nondescendants gven ts mmedate predecessors Drected acyclc graph MLDM-Berln hen 35
36 Example :Dentst Networ opology of networ encodes condtonal ndependence assertons Weather s ndependent of the other varables oothache and atch are condtonally ndependent gven avty avty s a drect cause of oothache and atch MLDM-Berln hen 36
37 ondtonal (In)dependence Defnton: X s condtonally ndependent of Y gven Z f the probablty dstrbuton governng X s ndependent of the value of Y gven the value of Z; that s f ( x y z ) ( X x Y y Z z ) ( X x Z z ) More compactly we can wrte ( X Y Z ) ( X Z ) ondtonal ndependence allows breang down nference nto calculaton over small group of varables MLDM-Berln hen 37
38 ondtonal (In)dependence (cont.) Example: hunder s condtonally ndependent of Ran gven Lghtnng (hunder Ran Lghtnng) (hunder Lghtnng) Recall that Naïve Bayes uses condtonal ndependence to ustfy ( X Y Z ) ( X Y Z ) ( Y Z ) ( X Z ) ( Y Z ) XY are mutually ndependent gven Z MLDM-Berln hen 38
39 ondtonal (In)dependence (cont.) Bayesan Networ also can be thought of as a causal graph n that llustrates causaltes between varables We can mae a dagnostc nference from the t ( R W )? predcton dagnoss ( R W ) ( R) ( R) W ( W ) ( W R) ( R) ( R) ( R) + ( W R) ( R) W ( > ( R) 0.4) MLDM-Berln hen 39
40 ondtonal (In)dependence (cont.) Suppose that sprnler s ncluded as another cause of wet grass redctve nference ( W S ) ( W R S ) ( R S ) + ( W R S ) ( R S ) ( W R S ) ( R) + ( W R S ) ( R) ( W ) ( W R S ) ( R S ) + ( W R S ) ( R S ) + ( W R S ) ( R S ) + ( W R S ) ( R S ) ( W R S ) ( R ) ( S ) + ( W R S ) ( R ) ( S ) + ( W R S ) ( R ) ( S ) + ( W R S ) ( R ) ( S ) Dagnostc nference (I) ( S W ) ( W S ) ( S ) ( W ) ( > ( S ) 0.2 ) MLDM-Berln hen 40
41 ondtonal (In)dependence (cont.) Dagnostc nference (II) S R W ( W R S ) ( S R ) ( W R ) ( W R S ) ( S ) ( W R ) ( > ( S ) 0. 2 ) ( W R ) ( W R S ) ( S R ) + ( W R S ) ( S R ) ( W R S ) ( S ) + ( W R S ) ( S ) MLDM-Berln hen 4
42 Example 2: Burglary Networ You re at wor neghbor John calls to say your alarm s rngng but neghbor Mary doesn't call. Sometmes t's set off by mnor earthquaes. Is there a burglar? ( Burglary Johnall Maryall F )? Varables: Burglar Earthquae Alarm Johnalls Maryalls Networ topology reflects causal nowledge A burglar can set the alarm off An earthquae can set the alarm off he alarm can cause Mary to call he alarm can cause John to call But John sometmes confuses the telephone rngng wth the alarm Mary les rather loud musc and sometmes msses the alarm MLDM-Berln hen 42
43 Example 2: Burglary Networ ondtonal robablty able () each row shows prob. gven a state of parents For Boolean varables ust the prob. for true s shown MLDM-Berln hen 43
44 ompactness han rule A for Boolean X wth Boolean (true/false) parents has 2 rows for the combnatons of parent values Each row requres one number p for X true (the number for X false s ust -p) If each varable has no more than parents the complete networ requres O(n 2 ) numbers I.e. grows lnearly wth n vs. O(2 n ) for the full ont dstrbuton For burglary net numbers (vs ?) ( B E A J M ) ( B) ( E B) ( A B E) ( J B E A) ( M B E A J ) ( B) ( E) ( A B E) ( J A) ( M A) MLDM-Berln hen 44
45 Global Semantcs Global semantcs defnes the full ont dstrbuton as the product of the local condtonal dstrbutons n ( X X ) X arents( X ) n... he Bayesan Networ s semantcally A representaton of the ont dstrbuton A encodng of a collecton of condtonal ndependence statements E.g. ( J ( J M A B E) A) ( M A) ( A B E) ( B) ( E) MLDM-Berln hen 45
46 Local Semantcs Local semantcs: each node s condtonally ndependent of ts nondescendants gven ts parents Local semantcs global semantcs MLDM-Berln hen 46
47 Marov Blanet Each node s condtonally ndependent of all others gven ts parents + chldren + chldren's parents MLDM-Berln hen 47
48 onstructng Bayesan Networs Need a method such that a seres of locally testable assertons of condtonal ndependence guarantees the requred global semantcs. hoose an orderng of varables X.. X.. X n 2. For to n add X to the networ and select parents from X such.. X that arents( X ) { X X } ( X X X ) ( X arents( X )).. hs choce of parents guarantees the global semantcs n ( X... X n ) ( X X.. X ) n ( X arents( X )) (chan rule).. (by constructon) MLDM-Berln hen 48
49 Example for onstructng Bayesan Networ Suppose we choose the orderng: M J A B E (J M) (J)? MLDM-Berln hen 49
50 Example (cont.) Suppose we choose the orderng: M J A B E (J M) (J)? No (A JM) (A J)? (A JM) (A)? MLDM-Berln hen 50
51 Example (cont.) Suppose we choose the orderng: M J A B E (J M) (J)? No (A JM) (A J)? No (A JM) (A)? No (B AJM) (B A)? (B AJM) (B)? MLDM-Berln hen 5
52 Example (cont.) Suppose we choose the orderng: M J A B E (J M) (J)? No (A JM) (A J)? No (A JM) (A)? No (B AJM) (B A)? Yes (B AJM) (B)? No (E BAJM) (E A)? (E BAJM) (E BA)? MLDM-Berln hen 52
53 Example (cont.) Suppose we choose the orderng: M J A B E (J M) (J)? No (A JM) (A J)? No (A JM) (A)? No (B AJM) (B A)? Yes (B AJM) (B)? No (E BAJM) (E A)? No (E BAJM) (E BA)? Yes MLDM-Berln hen 53
54 Example (cont.) Summary Decdng condtonal ndependence s hard n noncausal drectons (ausal models and condtonal ndependence seem hardwred for humans!) Assessng condtonal probabltes s hard n noncausal drectons Networ s less compact: numbers needed MLDM-Berln hen 54
55 Inference ass Smple queres: compute posteror margnal E.g. ( Burglary Johnalls true Marryalls true) onunctve queres: X X E e X X E e ( X E e) Optmal decsons: probablstc nference ( Outcome Acton Evdence) ( X E e) MLDM-Berln hen 55
56 MLDM-Berln hen 56 Inference by Enumeraton Slghtly ntellgent way to sum out varables from the ont wthout actually constructng ts explct representaton Smple query on the burglary networ Rewrte full ont entres usng product of entres: m a a e B a e B m a a e B a e B m a e B m B e a e a e a α α α e a m a e B m m B m m B m B ) ( α α α
57 Evaluaton ree Enumeraton s neffcent: repeated computaton\al e E.g. computes a m a for each value of MLDM-Berln hen 57
58 HW-4: Bayesan Networs A new bnary varable concernng cat mang nose on the roof (roof) S R redctve nferences ( F )? W F ( F S )? MLDM-Berln hen 58
59 MLDM-Berln hen 59 Bayesan Networs for Informaton Retreval D W t w d t d w d d w t w d t d t w d t d w t d w d Documents opcs Words
60 MLDM-Berln hen 60 Bayesan Networs for Informaton Retreval D W t w d t t w d t d d w d d w t w t d t w t d w d Documents opcs Words
CHAPTER 3: BAYESIAN DECISION THEORY
HATER 3: BAYESIAN DEISION THEORY Decson mang under uncertanty 3 Data comes from a process that s completely not nown The lac of nowledge can be compensated by modelng t as a random process May be the underlyng
More informationDepartment of Computer Science Artificial Intelligence Research Laboratory. Iowa State University MACHINE LEARNING
MACHINE LEANING Vasant Honavar Bonformatcs and Computatonal Bology rogram Center for Computatonal Intellgence, Learnng, & Dscovery Iowa State Unversty honavar@cs.astate.edu www.cs.astate.edu/~honavar/
More informationEvaluation for sets of classes
Evaluaton for Tet Categorzaton Classfcaton accuracy: usual n ML, the proporton of correct decsons, Not approprate f the populaton rate of the class s low Precson, Recall and F 1 Better measures 21 Evaluaton
More informationBayesian Learning. Smart Home Health Analytics Spring Nirmalya Roy Department of Information Systems University of Maryland Baltimore County
Smart Home Health Analytcs Sprng 2018 Bayesan Learnng Nrmalya Roy Department of Informaton Systems Unversty of Maryland Baltmore ounty www.umbc.edu Bayesan Learnng ombnes pror knowledge wth evdence to
More informationArtificial Intelligence Bayesian Networks
Artfcal Intellgence Bayesan Networks Adapted from sldes by Tm Fnn and Mare desjardns. Some materal borrowed from Lse Getoor. 1 Outlne Bayesan networks Network structure Condtonal probablty tables Condtonal
More informationClassification Bayesian Classifiers
lassfcaton Bayesan lassfers Jeff Howbert Introducton to Machne Learnng Wnter 2014 1 Bayesan classfcaton A robablstc framework for solvng classfcaton roblems. Used where class assgnment s not determnstc,.e.
More informationMIMA Group. Chapter 2 Bayesian Decision Theory. School of Computer Science and Technology, Shandong University. Xin-Shun SDU
Group M D L M Chapter Bayesan Decson heory Xn-Shun Xu @ SDU School of Computer Scence and echnology, Shandong Unversty Bayesan Decson heory Bayesan decson theory s a statstcal approach to data mnng/pattern
More informationBayesian Networks. Course: CS40022 Instructor: Dr. Pallab Dasgupta
Bayesan Networks Course: CS40022 Instructor: Dr. Pallab Dasgupta Department of Computer Scence & Engneerng Indan Insttute of Technology Kharagpur Example Burglar alarm at home Farly relable at detectng
More information2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification
E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton
More informationBayesian decision theory. Nuno Vasconcelos ECE Department, UCSD
Bayesan decson theory Nuno Vasconcelos ECE Department UCSD Notaton the notaton n DHS s qute sloppy e.. show that error error z z dz really not clear what ths means we wll use the follown notaton subscrpts
More informationGenerative classification models
CS 675 Intro to Machne Learnng Lecture Generatve classfcaton models Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Data: D { d, d,.., dn} d, Classfcaton represents a dscrete class value Goal: learn
More informationLogistic Regression. CAP 5610: Machine Learning Instructor: Guo-Jun QI
Logstc Regresson CAP 561: achne Learnng Instructor: Guo-Jun QI Bayes Classfer: A Generatve model odel the posteror dstrbuton P(Y X) Estmate class-condtonal dstrbuton P(X Y) for each Y Estmate pror dstrbuton
More informationSpeech and Language Processing
Speech and Language rocessng Lecture 3 ayesan network and ayesan nference Informaton and ommuncatons Engneerng ourse Takahro Shnozak 08//5 Lecture lan (Shnozak s part) I gves the frst 6 lectures about
More information9.913 Pattern Recognition for Vision. Class IV Part I Bayesian Decision Theory Yuri Ivanov
9.93 Class IV Part I Bayesan Decson Theory Yur Ivanov TOC Roadmap to Machne Learnng Bayesan Decson Makng Mnmum Error Rate Decsons Mnmum Rsk Decsons Mnmax Crteron Operatng Characterstcs Notaton x - scalar
More informationHidden Markov Models
CM229S: Machne Learnng for Bonformatcs Lecture 12-05/05/2016 Hdden Markov Models Lecturer: Srram Sankararaman Scrbe: Akshay Dattatray Shnde Edted by: TBD 1 Introducton For a drected graph G we can wrte
More informationMachine learning: Density estimation
CS 70 Foundatons of AI Lecture 3 Machne learnng: ensty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square ata: ensty estmaton {.. n} x a vector of attrbute values Objectve: estmate the model of
More informationINF 5860 Machine learning for image classification. Lecture 3 : Image classification and regression part II Anne Solberg January 31, 2018
INF 5860 Machne learnng for mage classfcaton Lecture 3 : Image classfcaton and regresson part II Anne Solberg January 3, 08 Today s topcs Multclass logstc regresson and softma Regularzaton Image classfcaton
More information10-701/ Machine Learning, Fall 2005 Homework 3
10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40
More informationCIS587 - Artificial Intellgence. Bayesian Networks CIS587 - AI. KB for medical diagnosis. Example.
CIS587 - Artfcal Intellgence Bayesan Networks KB for medcal dagnoss. Example. We want to buld a KB system for the dagnoss of pneumona. Problem descrpton: Dsease: pneumona Patent symptoms (fndngs, lab tests):
More informationWhy Bayesian? 3. Bayes and Normal Models. State of nature: class. Decision rule. Rev. Thomas Bayes ( ) Bayes Theorem (yes, the famous one)
Why Bayesan? 3. Bayes and Normal Models Alex M. Martnez alex@ece.osu.edu Handouts Handoutsfor forece ECE874 874Sp Sp007 If all our research (n PR was to dsappear and you could only save one theory, whch
More informationReasoning under Uncertainty
Reasonng under Uncertanty Course: CS40022 Instructor: Dr. Pallab Dasgupta Department of Computer Scence & Engneerng Indan Insttute of Technology Kharagpur Handlng uncertan knowledge p p Symptom(p, Toothache
More informationKernel Methods and SVMs Extension
Kernel Methods and SVMs Extenson The purpose of ths document s to revew materal covered n Machne Learnng 1 Supervsed Learnng regardng support vector machnes (SVMs). Ths document also provdes a general
More informationCS 2750 Machine Learning. Lecture 5. Density estimation. CS 2750 Machine Learning. Announcements
CS 750 Machne Learnng Lecture 5 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square CS 750 Machne Learnng Announcements Homework Due on Wednesday before the class Reports: hand n before
More informationThe Gaussian classifier. Nuno Vasconcelos ECE Department, UCSD
he Gaussan classfer Nuno Vasconcelos ECE Department, UCSD Bayesan decson theory recall that we have state of the world X observatons g decson functon L[g,y] loss of predctng y wth g Bayes decson rule s
More informationINTRODUCTION TO MACHINE LEARNING 3RD EDITION
ETHEM ALPAYDIN The MIT Press, 2014 Lecture Sldes for INTRODUCTION TO MACHINE LEARNING 3RD EDITION alpaydn@boun.edu.tr http://www.cmpe.boun.edu.tr/~ethem/2ml3e CHAPTER 3: BAYESIAN DECISION THEORY Probablty
More informationEvaluation of classifiers MLPs
Lecture Evaluaton of classfers MLPs Mlos Hausrecht mlos@cs.ptt.edu 539 Sennott Square Evaluaton For any data set e use to test the model e can buld a confuson matrx: Counts of examples th: class label
More informationProbability review. Adopted from notes of Andrew W. Moore and Eric Xing from CMU. Copyright Andrew W. Moore Slide 1
robablty reew dopted from notes of ndrew W. Moore and Erc Xng from CMU Copyrght ndrew W. Moore Slde So far our classfers are determnstc! For a gen X, the classfers we learned so far ge a sngle predcted
More informationP R. Lecture 4. Theory and Applications of Pattern Recognition. Dept. of Electrical and Computer Engineering /
Theory and Applcatons of Pattern Recognton 003, Rob Polkar, Rowan Unversty, Glassboro, NJ Lecture 4 Bayes Classfcaton Rule Dept. of Electrcal and Computer Engneerng 0909.40.0 / 0909.504.04 Theory & Applcatons
More informationENG 8801/ Special Topics in Computer Engineering: Pattern Recognition. Memorial University of Newfoundland Pattern Recognition
EG 880/988 - Specal opcs n Computer Engneerng: Pattern Recognton Memoral Unversty of ewfoundland Pattern Recognton Lecture 7 May 3, 006 http://wwwengrmunca/~charlesr Offce Hours: uesdays hursdays 8:30-9:30
More informationBayesian belief networks
CS 1571 Introducton to I Lecture 24 ayesan belef networks los Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square CS 1571 Intro to I dmnstraton Homework assgnment 10 s out and due next week Fnal exam: December
More informationCS 3710: Visual Recognition Classification and Detection. Adriana Kovashka Department of Computer Science January 13, 2015
CS 3710: Vsual Recognton Classfcaton and Detecton Adrana Kovashka Department of Computer Scence January 13, 2015 Plan for Today Vsual recognton bascs part 2: Classfcaton and detecton Adrana s research
More informationClassification as a Regression Problem
Target varable y C C, C,, ; Classfcaton as a Regresson Problem { }, 3 L C K To treat classfcaton as a regresson problem we should transform the target y nto numercal values; The choce of numercal class
More informationA Bayes Algorithm for the Multitask Pattern Recognition Problem Direct Approach
A Bayes Algorthm for the Multtask Pattern Recognton Problem Drect Approach Edward Puchala Wroclaw Unversty of Technology, Char of Systems and Computer etworks, Wybrzeze Wyspanskego 7, 50-370 Wroclaw, Poland
More information1 Convex Optimization
Convex Optmzaton We wll consder convex optmzaton problems. Namely, mnmzaton problems where the objectve s convex (we assume no constrants for now). Such problems often arse n machne learnng. For example,
More informationOther NN Models. Reinforcement learning (RL) Probabilistic neural networks
Other NN Models Renforcement learnng (RL) Probablstc neural networks Support vector machne (SVM) Renforcement learnng g( (RL) Basc deas: Supervsed dlearnng: (delta rule, BP) Samples (x, f(x)) to learn
More informationStatistical pattern recognition
Statstcal pattern recognton Bayes theorem Problem: decdng f a patent has a partcular condton based on a partcular test However, the test s mperfect Someone wth the condton may go undetected (false negatve
More informationUsing T.O.M to Estimate Parameter of distributions that have not Single Exponential Family
IOSR Journal of Mathematcs IOSR-JM) ISSN: 2278-5728. Volume 3, Issue 3 Sep-Oct. 202), PP 44-48 www.osrjournals.org Usng T.O.M to Estmate Parameter of dstrbutons that have not Sngle Exponental Famly Jubran
More informationHomework Assignment 3 Due in class, Thursday October 15
Homework Assgnment 3 Due n class, Thursday October 15 SDS 383C Statstcal Modelng I 1 Rdge regresson and Lasso 1. Get the Prostrate cancer data from http://statweb.stanford.edu/~tbs/elemstatlearn/ datasets/prostate.data.
More informationRetrieval Models: Language models
CS-590I Informaton Retreval Retreval Models: Language models Luo S Department of Computer Scence Purdue Unversty Introducton to language model Ungram language model Document language model estmaton Maxmum
More informationNeural networks. Nuno Vasconcelos ECE Department, UCSD
Neural networs Nuno Vasconcelos ECE Department, UCSD Classfcaton a classfcaton problem has two types of varables e.g. X - vector of observatons (features) n the world Y - state (class) of the world x X
More informationClassification learning II
Lecture 8 Classfcaton learnng II Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Logstc regresson model Defnes a lnear decson boundar Dscrmnant functons: g g g g here g z / e z f, g g - s a logstc functon
More informationBayesian classification CISC 5800 Professor Daniel Leeds
Tran Test Introducton to classfers Bayesan classfcaton CISC 58 Professor Danel Leeds Goal: learn functon C to maxmze correct labels (Y) based on features (X) lon: 6 wolf: monkey: 4 broker: analyst: dvdend:
More informationOutline. Bayesian Networks: Maximum Likelihood Estimation and Tree Structure Learning. Our Model and Data. Outline
Outlne Bayesan Networks: Maxmum Lkelhood Estmaton and Tree Structure Learnng Huzhen Yu janey.yu@cs.helsnk.f Dept. Computer Scence, Unv. of Helsnk Probablstc Models, Sprng, 200 Notces: I corrected a number
More informationCIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M
CIS56: achne Learnng Lecture 3 (Sept 6, 003) Preparaton help: Xaoyng Huang Lnear Regresson Lnear regresson can be represented by a functonal form: f(; θ) = θ 0 0 +θ + + θ = θ = 0 ote: 0 s a dummy attrbute
More informationMarkov Chain Monte Carlo (MCMC), Gibbs Sampling, Metropolis Algorithms, and Simulated Annealing Bioinformatics Course Supplement
Markov Chan Monte Carlo MCMC, Gbbs Samplng, Metropols Algorthms, and Smulated Annealng 2001 Bonformatcs Course Supplement SNU Bontellgence Lab http://bsnuackr/ Outlne! Markov Chan Monte Carlo MCMC! Metropols-Hastngs
More informationFor now, let us focus on a specific model of neurons. These are simplified from reality but can achieve remarkable results.
Neural Networks : Dervaton compled by Alvn Wan from Professor Jtendra Malk s lecture Ths type of computaton s called deep learnng and s the most popular method for many problems, such as computer vson
More informationCS47300: Web Information Search and Management
CS47300: Web Informaton Search and Management Probablstc Retreval Models Prof. Chrs Clfton 7 September 2018 Materal adapted from course created by Dr. Luo S, now leadng Albaba research group 14 Why probabltes
More informationLecture 19 of 42. MAP and MLE continued, Minimum Description Length (MDL)
Lecture 19 of 4 MA and MLE contnued, Mnu Descrpton Length (MDL) Wednesday, 8 February 007 Wlla H. Hsu, KSU http://www.kddresearch.org Readngs for next class: Chapter 5, Mtchell Lecture Outlne Read Sectons
More informationLearning from Data 1 Naive Bayes
Learnng from Data 1 Nave Bayes Davd Barber dbarber@anc.ed.ac.uk course page : http://anc.ed.ac.uk/ dbarber/lfd1/lfd1.html c Davd Barber 2001, 2002 1 Learnng from Data 1 : c Davd Barber 2001,2002 2 1 Why
More informationChapter 9: Statistical Inference and the Relationship between Two Variables
Chapter 9: Statstcal Inference and the Relatonshp between Two Varables Key Words The Regresson Model The Sample Regresson Equaton The Pearson Correlaton Coeffcent Learnng Outcomes After studyng ths chapter,
More information} Often, when learning, we deal with uncertainty:
Uncertanty and Learnng } Often, when learnng, we deal wth uncertanty: } Incomplete data sets, wth mssng nformaton } Nosy data sets, wth unrelable nformaton } Stochastcty: causes and effects related non-determnstcally
More informationEM and Structure Learning
EM and Structure Learnng Le Song Machne Learnng II: Advanced Topcs CSE 8803ML, Sprng 2012 Partally observed graphcal models Mxture Models N(μ 1, Σ 1 ) Z X N N(μ 2, Σ 2 ) 2 Gaussan mxture model Consder
More informationStatistics for Economics & Business
Statstcs for Economcs & Busness Smple Lnear Regresson Learnng Objectves In ths chapter, you learn: How to use regresson analyss to predct the value of a dependent varable based on an ndependent varable
More informationSupport Vector Machines. Vibhav Gogate The University of Texas at dallas
Support Vector Machnes Vbhav Gogate he Unversty of exas at dallas What We have Learned So Far? 1. Decson rees. Naïve Bayes 3. Lnear Regresson 4. Logstc Regresson 5. Perceptron 6. Neural networks 7. K-Nearest
More informationMDL-Based Unsupervised Attribute Ranking
MDL-Based Unsupervsed Attrbute Rankng Zdravko Markov Computer Scence Department Central Connectcut State Unversty New Brtan, CT 06050, USA http://www.cs.ccsu.edu/~markov/ markovz@ccsu.edu MDL-Based Unsupervsed
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Maxmum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models
More informationj) = 1 (note sigma notation) ii. Continuous random variable (e.g. Normal distribution) 1. density function: f ( x) 0 and f ( x) dx = 1
Random varables Measure of central tendences and varablty (means and varances) Jont densty functons and ndependence Measures of assocaton (covarance and correlaton) Interestng result Condtonal dstrbutons
More informationANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)
Econ 413 Exam 13 H ANSWERS Settet er nndelt 9 deloppgaver, A,B,C, som alle anbefales å telle lkt for å gøre det ltt lettere å stå. Svar er gtt . Unfortunately, there s a prntng error n the hnt of
More informationWhat Independencies does a Bayes Net Model? Bayesian Networks: Independencies and Inference. Quick proof that independence is symmetric
Bayesan Networks: Indeendences and Inference Scott Daves and ndrew Moore Note to other teachers and users of these sldes. ndrew and Scott would be delghted f you found ths source materal useful n gvng
More informationMaximum Likelihood Estimation (MLE)
Maxmum Lkelhood Estmaton (MLE) Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 175A Wnter 01 UCSD Statstcal Learnng Goal: Gven a relatonshp between a feature vector x and a vector y, and d data samples (x,y
More informationLimited Dependent Variables
Lmted Dependent Varables. What f the left-hand sde varable s not a contnuous thng spread from mnus nfnty to plus nfnty? That s, gven a model = f (, β, ε, where a. s bounded below at zero, such as wages
More information1. Inference on Regression Parameters a. Finding Mean, s.d and covariance amongst estimates. 2. Confidence Intervals and Working Hotelling Bands
Content. Inference on Regresson Parameters a. Fndng Mean, s.d and covarance amongst estmates.. Confdence Intervals and Workng Hotellng Bands 3. Cochran s Theorem 4. General Lnear Testng 5. Measures of
More informationMachine Learning. Classification. Theory of Classification and Nonparametric Classifier. Representing data: Hypothesis (classifier) Eric Xing
Machne Learnng 0-70/5 70/5-78, 78, Fall 008 Theory of Classfcaton and Nonarametrc Classfer Erc ng Lecture, Setember 0, 008 Readng: Cha.,5 CB and handouts Classfcaton Reresentng data: M K Hyothess classfer
More informationCell Biology. Lecture 1: 10-Oct-12. Marco Grzegorczyk. (Gen-)Regulatory Network. Microarray Chips. (Gen-)Regulatory Network. (Gen-)Regulatory Network
5.0.202 Genetsche Netzwerke Wntersemester 202/203 ell ology Lecture : 0-Oct-2 Marco Grzegorczyk Gen-Regulatory Network Mcroarray hps G G 2 G 3 2 3 metabolte metabolte Gen-Regulatory Network Gen-Regulatory
More informationI529: Machine Learning in Bioinformatics (Spring 2017) Markov Models
I529: Machne Learnng n Bonformatcs (Sprng 217) Markov Models Yuzhen Ye School of Informatcs and Computng Indana Unversty, Bloomngton Sprng 217 Outlne Smple model (frequency & profle) revew Markov chan
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationNatural Language Processing and Information Retrieval
Natural Language Processng and Informaton Retreval Support Vector Machnes Alessandro Moschtt Department of nformaton and communcaton technology Unversty of Trento Emal: moschtt@ds.untn.t Summary Support
More informationSpace of ML Problems. CSE 473: Artificial Intelligence. Parameter Estimation and Bayesian Networks. Learning Topics
/7/7 CSE 73: Artfcal Intellgence Bayesan - Learnng Deter Fox Sldes adapted from Dan Weld, Jack Breese, Dan Klen, Daphne Koller, Stuart Russell, Andrew Moore & Luke Zettlemoyer What s Beng Learned? Space
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationPattern Classification
Pattern Classfcaton All materals n these sldes ere taken from Pattern Classfcaton (nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wley & Sons, 000 th the permsson of the authors and the publsher
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More informationThe big picture. Outline
The bg pcture Vncent Claveau IRISA - CNRS, sldes from E. Kjak INSA Rennes Notatons classes: C = {ω = 1,.., C} tranng set S of sze m, composed of m ponts (x, ω ) per class ω representaton space: R d (=
More informationMultilayer Perceptron (MLP)
Multlayer Perceptron (MLP) Seungjn Cho Department of Computer Scence and Engneerng Pohang Unversty of Scence and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjn@postech.ac.kr 1 / 20 Outlne
More informationMAXIMUM A POSTERIORI TRANSDUCTION
MAXIMUM A POSTERIORI TRANSDUCTION LI-WEI WANG, JU-FU FENG School of Mathematcal Scences, Peng Unversty, Bejng, 0087, Chna Center for Informaton Scences, Peng Unversty, Bejng, 0087, Chna E-MIAL: {wanglw,
More informationVQ widely used in coding speech, image, and video
at Scalar quantzers are specal cases of vector quantzers (VQ): they are constraned to look at one sample at a tme (memoryless) VQ does not have such constrant better RD perfomance expected Source codng
More informationWhich Separator? Spring 1
Whch Separator? 6.034 - Sprng 1 Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng 3 Margn of a pont " # y (w $ + b) proportonal
More informationStatistical Foundations of Pattern Recognition
Statstcal Foundatons of Pattern Recognton Learnng Objectves Bayes Theorem Decson-mang Confdence factors Dscrmnants The connecton to neural nets Statstcal Foundatons of Pattern Recognton NDE measurement
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Mamum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models for
More informationChapter 8 Indicator Variables
Chapter 8 Indcator Varables In general, e explanatory varables n any regresson analyss are assumed to be quanttatve n nature. For example, e varables lke temperature, dstance, age etc. are quanttatve n
More informationCSC321 Tutorial 9: Review of Boltzmann machines and simulated annealing
CSC321 Tutoral 9: Revew of Boltzmann machnes and smulated annealng (Sldes based on Lecture 16-18 and selected readngs) Yue L Emal: yuel@cs.toronto.edu Wed 11-12 March 19 Fr 10-11 March 21 Outlne Boltzmann
More informationMLE and Bayesian Estimation. Jie Tang Department of Computer Science & Technology Tsinghua University 2012
MLE and Bayesan Estmaton Je Tang Department of Computer Scence & Technology Tsnghua Unversty 01 1 Lnear Regresson? As the frst step, we need to decde how we re gong to represent the functon f. One example:
More informationWeek 5: Neural Networks
Week 5: Neural Networks Instructor: Sergey Levne Neural Networks Summary In the prevous lecture, we saw how we can construct neural networks by extendng logstc regresson. Neural networks consst of multple
More informationPredictive Analytics : QM901.1x Prof U Dinesh Kumar, IIMB. All Rights Reserved, Indian Institute of Management Bangalore
Sesson Outlne Introducton to classfcaton problems and dscrete choce models. Introducton to Logstcs Regresson. Logstc functon and Logt functon. Maxmum Lkelhood Estmator (MLE) for estmaton of LR parameters.
More informationProbabilistic Classification: Bayes Classifiers. Lecture 6:
Probablstc Classfcaton: Bayes Classfers Lecture : Classfcaton Models Sam Rowes January, Generatve model: p(x, y) = p(y)p(x y). p(y) are called class prors. p(x y) are called class condtonal feature dstrbutons.
More informationProblem Set 9 Solutions
Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem
More informationRepresenting arbitrary probability distributions Inference. Exact inference; Approximate inference
Bayesan Learnng So far What does t mean to be Bayesan? Naïve Bayes Independence assumptons EM Algorthm Learnng wth hdden varables Today: Representng arbtrary probablty dstrbutons Inference Exact nference;
More informationMarginal Effects in Probit Models: Interpretation and Testing. 1. Interpreting Probit Coefficients
ECON 5 -- NOE 15 Margnal Effects n Probt Models: Interpretaton and estng hs note ntroduces you to the two types of margnal effects n probt models: margnal ndex effects, and margnal probablty effects. It
More informationCommunication with AWGN Interference
Communcaton wth AWG Interference m {m } {p(m } Modulator s {s } r=s+n Recever ˆm AWG n m s a dscrete random varable(rv whch takes m wth probablty p(m. Modulator maps each m nto a waveform sgnal s m=m
More informationFinite Mixture Models and Expectation Maximization. Most slides are from: Dr. Mario Figueiredo, Dr. Anil Jain and Dr. Rong Jin
Fnte Mxture Models and Expectaton Maxmzaton Most sldes are from: Dr. Maro Fgueredo, Dr. Anl Jan and Dr. Rong Jn Recall: The Supervsed Learnng Problem Gven a set of n samples X {(x, y )},,,n Chapter 3 of
More informationDepartment of Quantitative Methods & Information Systems. Time Series and Their Components QMIS 320. Chapter 6
Department of Quanttatve Methods & Informaton Systems Tme Seres and Ther Components QMIS 30 Chapter 6 Fall 00 Dr. Mohammad Zanal These sldes were modfed from ther orgnal source for educatonal purpose only.
More informationHere is the rationale: If X and y have a strong positive relationship to one another, then ( x x) will tend to be positive when ( y y)
Secton 1.5 Correlaton In the prevous sectons, we looked at regresson and the value r was a measurement of how much of the varaton n y can be attrbuted to the lnear relatonshp between y and x. In ths secton,
More informationDecision-making and rationality
Reslence Informatcs for Innovaton Classcal Decson Theory RRC/TMI Kazuo URUTA Decson-makng and ratonalty What s decson-makng? Methodology for makng a choce The qualty of decson-makng determnes success or
More informationThe Expectation-Maximization Algorithm
The Expectaton-Maxmaton Algorthm Charles Elan elan@cs.ucsd.edu November 16, 2007 Ths chapter explans the EM algorthm at multple levels of generalty. Secton 1 gves the standard hgh-level verson of the algorthm.
More informationLecture 3: Probability Distributions
Lecture 3: Probablty Dstrbutons Random Varables Let us begn by defnng a sample space as a set of outcomes from an experment. We denote ths by S. A random varable s a functon whch maps outcomes nto the
More informationNumber of cases Number of factors Number of covariates Number of levels of factor i. Value of the dependent variable for case k
ANOVA Model and Matrx Computatons Notaton The followng notaton s used throughout ths chapter unless otherwse stated: N F CN Y Z j w W Number of cases Number of factors Number of covarates Number of levels
More informationLinear Regression Analysis: Terminology and Notation
ECON 35* -- Secton : Basc Concepts of Regresson Analyss (Page ) Lnear Regresson Analyss: Termnology and Notaton Consder the generc verson of the smple (two-varable) lnear regresson model. It s represented
More informationChapter Newton s Method
Chapter 9. Newton s Method After readng ths chapter, you should be able to:. Understand how Newton s method s dfferent from the Golden Secton Search method. Understand how Newton s method works 3. Solve
More informationEdge Isoperimetric Inequalities
November 7, 2005 Ross M. Rchardson Edge Isopermetrc Inequaltes 1 Four Questons Recall that n the last lecture we looked at the problem of sopermetrc nequaltes n the hypercube, Q n. Our noton of boundary
More informationNP-Completeness : Proofs
NP-Completeness : Proofs Proof Methods A method to show a decson problem Π NP-complete s as follows. (1) Show Π NP. (2) Choose an NP-complete problem Π. (3) Show Π Π. A method to show an optmzaton problem
More informationNaïve Bayes Classifier
9/8/07 MIST.6060 Busness Intellgence and Data Mnng Naïve Bayes Classfer Termnology Predctors: the attrbutes (varables) whose values are used for redcton and classfcaton. Predctors are also called nut varables,
More information