Machine Learning. knowledge acquisition skill refinement. Relation between machine learning and data mining. P. Berka, /18
|
|
- Barnaby Thornton
- 5 years ago
- Views:
Transcription
1 Mache Learg The feld of mache learg s cocered wth the questo of how to costruct computer programs that automatcally mprove wth eperece. (Mtchell, 1997) Thgs lear whe they chage ther behavor a way that makes them perform better a future. (Wtte, Frak, 1999) types of learg: kowledge acqusto skll refemet Relato betwee mache learg ad data mg P. Berka, /18
2 learg decso makg select represetato learg kowledge object object decso decso descrpto makg Geeral scheme of a learg system Learg methods: 1. rote learg, 2. learg from structo, learg by beg told), 3. learg by aalogy, stace-based learg, lazy learg, 4. eplaato-based learg, 5. learg from eamples, 6. learg from observato ad dscovery. P. Berka, /18
3 Learg methods: statstcal methods - regresso methods, dscrmat aalyss, cluster aalyss, symbolc mache learg methods - decso trees ad rules, case-based reasog (CBR) sub-symbolc mache learg methods euroal etworks, bayesa etworks or geetc algorthms. Feedback durg learg: pre-classfed eamples (supervsed learg) small umber of pre-classfed eamples ad a large umber of eamples wthout kow class (sem-supervsed learg) algorthm ca query the teacher for class membershp for uclassfed eamples (actve learg), drect hts derved from the teacher s behavor (appretceshp learg) o feedback (usupervsed learg) P. Berka, /18
4 represetato of eamples: 1. attrbutes: categoral (bary, omal, ordal) ad umerc [har=black & heght=180 & beard=yes & educato=uv] 2. relatos father(ja_lucembursky, karel_iv) Algorthms: batch all eamples are processed at oce cremetal eamples are processed subsequetly system ca be re-traed Learg methods: emprcal uses large set of (trag) eamples ad lmted (or o) backgroud kowledge aalytc uses large backgroud kowledge ad several (oe or eve o) llustratve eamples P. Berka, /18
5 Prcples of emprcal cocept learg 1. eamples of the same class have smlar characterstcs (smlarty-based learg) 1. eamples of the same class create clusters the attrbute space The goal of learg s to fd ad represet these clusters garbage, garbage out problem Importace of data uderstadg ad preprocessg P. Berka, /18
6 2. Geeral kowledge ferred from a fte set of eamples (ductve learg) Eamples dvded to 2 (or 3) sets: o trag set to buld a model o (valdato set to tue the parameters) o testg set to test the model P. Berka, /18
7 Geeral defto of (supervsed) mache learg Aalyzed data: D : : m 2 m : m Rows the table represet objects (eamples, staces) Colums the table correspod to attrbutes (varables) Whe addg target attrbute to the data table, we obta data sutable for supervsed learg methods (so called trag data). D TR : : m 2 m : m y y y : 1 2 Classfcato task: to fd kowledge (represeted by a decso fucto f), that assgs value of target attrbute y to a object descrbed by values of put attrbutes f: y. P. Berka, /18
8 We fer durg classfcato for values of put attrbutes for a object the value of target attrbutes: ŷ = f (). The derved value ŷ ca be dfferet from the real value y. We ca thus compute for every object o D TR the classfcato error Q f (o, ŷ ). for umerc attrbute C e.g. as: Q (, y ) = (y - y ) 2 f o for categoral attrbute C e.g. as: Q f ( o, yˆ 1 ) = 0 ff ff y y yˆ = yˆ We ca compute the overall error Err(f,D TR ) for the whole trag set D TR e.g. as mea error:. =1 Err(f,D = 1 TR ) Q f ( o, y ) The goal of learg s to fd such kowledge f*, that wll mmze ths error Err(f*,D TR ) m Err(f,DTR ). f P. Berka, /18
9 1. Learg as search Lookg for both structure ad parameters of the model Models as cluster descrptos: MGM most geeral model (oe cluster for all eamples) MSM most specfc model(s) (each eample creates a cluster) M1 s more geeral tha M2, M2 s more specfc tha M1 1 1 B( ) B( k), B(0) 1 k k B( ) Bell umbers P. Berka, /
10 Search methods: Drecto Top dow (from geeral to specfc models) Bottom up (from specfc to geeral models) Strategy bld (we cosder each possblty how to specalze/geeralze gve model) heurstc (we use some crtero to select oly the best possbltes how to specalze/geeralze gve model) radom Badwdth sgle (we cosder oly oe trasformato of actual model) parallel (we cosder more trasformatos) P. Berka, /18
11 Eample: Let us assume, that both put attrbutes ad target attrbute are categoral let us deote category the value of a attrbute: 1. atomc formula that epresses property of object o : A (v j k )( o ) 1 0 pro pro j j v v 2. set of objects that fulfll gve property A (v ) { o : } j k j v k k k Combatos are created from categores usg logcal AND Comb [ A (v ), A (v ),...,A (v )] A (v ) A (v )... A j1 k1 j2 k2 jl kl j1 k1 j2 k2 jl kl (v ) 1. 1 o Comb( o ) 0 f else v v... v j1 k1 j 2 k2 j l kl : 2. Comb { : vk j vk... j } o. j v l k l Comb covers object o ff Comb(o ) = 1 We ca create supercombatos by addg categores to a combato ad create subcombatos by removg categores from a combato. P. Berka, /18
12 Partal orderg betwee combatos: If combato Comb 1 s a subcombato of combato Comb 2, the combato Comb 1 s more geeral tha combato Comb 2 ad combato Comb 2 s more specfc tha combato Comb 1. If combato Comb 1 s more geeral tha combato Comb 2, the Comb 1 covers at least all objects that are covered by Comb 2. (dowward-closure property) The resultg kowledge wll be represeted by combatos that cover oly eamples of gve class. Combato Comb s cosstet, ff t covers oly eamples of a sgle class: C(v t ) o Eample data: D TR : Comb( o ) 1 y v příjem koto pohlaví ezaměstaý auto bydleí úvěr vysoký vysoké žea e ao vlastí Ao vysoký vysoké muž e ao vlastí Ao zký ízké muž e ao ájemí Ne vysoký vysoké muž e e ájemí Ao Combato Comb (hypothess represetg the cocept úvěr ) ca cota followg values of a attrbute: t? to dcate that the value of ths attrbute s rrelevat, value of the attrbute, to dcate that o value of ths attrbute s applcable. P. Berka, /18
13 [?,?,?,?,?,?]... [?,?, žea,?,?,?] [vysoký,?,?,?,?,?] [?,vysoké,?,?,?,?] [?,?,?,?,?, vlastí]... [vysoký,?,?, e,?,?] [vysoký,vysoké,?,?,?,?] [?, vysoké,?, e,?,?,?] [vysoký,vysoké,?, e,?,?] [vysoký,vysoké,?, e,ao,?] [vysoký,vysoké,?,e,?,vlastí] [vysoký,vysoké,muž, e,?,?] [vysoký, vysoké,?, e,ao, vlastí] [vysoký, vysoké, muž,e,?,vlastí] [vysoký, vysoké, žea,e,?,vlastí] [vysoký, vysoké, muž,e, ao,?] [vysoký, vysoké, muž,e, e,?] [vysoký,vysoké,žea,e, ao, vlastí] [vysoký,vysoké,muž,e, ao, vlastí] [vysoký,vysoké,muž,e, e, ájemí] [,,,,, ] Hypothess space We ca traverse the hypothess space usg two methods: from geeral to specfc (top-dow, specalzato), from specfc to geeral (bottom-up, geeralzato). P. Berka, /18
14 Fd-S algorthm 1. Italze h to the most specfc hypothess H 2. For each postve trag eample 2.1. For each attrbute a from hypothess h f value of attrbute a does ot correspod to the replace value of a by the et more geeral value that correspods to 3. output h S: [vysoký, vysoké,?,e,?,?] [vysoký,vysoké,?, e,ao,?] [vysoký,vysoké,?,e,?,vlastí] [vysoký,vysoké,muž, e,?,?] [vysoký, vysoké,?, e,ao, vlastí] [vysoký, vysoké, muž,e, e,?] [vysoký,vysoké,žea,e, ao, vlastí] [vysoký,vysoké,muž,e, ao, vlastí] [vysoký,vysoké,muž,e, e, ájemí] P. Berka, /18
15 Caddate-Elmato algorthm 1. Italze G to the set of mamally geeral hypotheses H 2. Italze S to the set of mamally specfc hypotheses H 3. for each eample 3.1. f s a postve eample the remove form G ay hypothess cosstet wth for each hypothess s S that s ot cosstet wth remove s from S add to S mmal geeralzato h of s such, that h s cosstet wth ad some member of G s more geeral tha h remove from S hypotheses that are more geeral tha aother hypothess S 3.2. f s a egatve eample the remove from S ay hypothess cosstet wth for each hypothess g G that s ot cosstet wth remove g from G add to G mmal specalzato h of g such, that h s cosstet wth ad some member of S s more specfc tha h remove from G hypotheses that are more specfc tha aother hypothess G G: [vysoký,?,?,?,?,?] [?, vysoké,?,?,?,?] [vysoký,?,?, e,?,?] [vysoký,vysoké,?,?,?,?] [?, vysoké,?, e,?,?,?] S: [vysoký, vysoké,?,e,?,?] P. Berka, /18
16 2. Learg as appromato Lookg oly for parameters of the model Eample: usg fte umber of data pots fd parameters of a (geeral) fucto to best ft the data y=f() f() = q 1 + q 0 least squares method: the problem of fdg the mmum of the overall error m (y - f( )) 2 s trasformed to solvg the equato d dq (y - f( )) 2 = 0 P. Berka, /18
17 soluto: 1) aalytc (we kow the type of fucto) solvg the equatos for the parameters of fuctos q 0 = ( ky k )( k k2 ) - ( k k y k )( k k ) ( k k2 ) - ( k k ) 2 q 1 = ( k k y k ) - ( k k )( k y k ) ( k k2 )- ( k k ) 2 2) umerc (we do ot kow the type of fucto) gradet methods Err(q) = Err q 0, Err q 1,..., Err q Q. Modfcato of kowledge q = [q 0, q 1,..., q Q ] accordg the algorthm where q j q j + q j Δq j Err - η q j ad s a parameter epressg step used to approach the mmum of fucto Err. P. Berka, /18
18 E.g. for error fucto 1 Err(f,D = (y - y 1 TR ) ) (y - f`( )) 2 2 =1 2 2 ad epected fucto f as lear combato of puts f() = q, =1 we ca derve the gradet of fucto Err as Err q So j = y - y~ = 2y - y~ y - y~ 1 q j 2 1 q j 1 y - y~ y - q = y - y~ - j = q 1 j 1 q = y - y j j =1 Problem wth covergece to local mmum P. Berka, /18
STA 108 Applied Linear Models: Regression Analysis Spring Solution for Homework #1
STA 08 Appled Lear Models: Regresso Aalyss Sprg 0 Soluto for Homework #. Let Y the dollar cost per year, X the umber of vsts per year. The the mathematcal relato betwee X ad Y s: Y 300 + X. Ths s a fuctoal
More informationBayes (Naïve or not) Classifiers: Generative Approach
Logstc regresso Bayes (Naïve or ot) Classfers: Geeratve Approach What do we mea by Geeratve approach: Lear p(y), p(x y) ad the apply bayes rule to compute p(y x) for makg predctos Ths s essetally makg
More informationCHAPTER VI Statistical Analysis of Experimental Data
Chapter VI Statstcal Aalyss of Expermetal Data CHAPTER VI Statstcal Aalyss of Expermetal Data Measuremets do ot lead to a uque value. Ths s a result of the multtude of errors (maly radom errors) that ca
More informationCSE 5526: Introduction to Neural Networks Linear Regression
CSE 556: Itroducto to Neural Netorks Lear Regresso Part II 1 Problem statemet Part II Problem statemet Part II 3 Lear regresso th oe varable Gve a set of N pars of data , appromate d by a lear fucto
More informationSupervised learning: Linear regression Logistic regression
CS 57 Itroducto to AI Lecture 4 Supervsed learg: Lear regresso Logstc regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Data: D { D D.. D D Supervsed learg d a set of eamples s
More informationBayesian Classification. CS690L Data Mining: Classification(2) Bayesian Theorem: Basics. Bayesian Theorem. Training dataset. Naïve Bayes Classifier
Baa Classfcato CS6L Data Mg: Classfcato() Referece: J. Ha ad M. Kamber, Data Mg: Cocepts ad Techques robablstc learg: Calculate explct probabltes for hypothess, amog the most practcal approaches to certa
More informationAn Introduction to. Support Vector Machine
A Itroducto to Support Vector Mache Support Vector Mache (SVM) A classfer derved from statstcal learg theory by Vapk, et al. 99 SVM became famous whe, usg mages as put, t gave accuracy comparable to eural-etwork
More informationUnsupervised Learning and Other Neural Networks
CSE 53 Soft Computg NOT PART OF THE FINAL Usupervsed Learg ad Other Neural Networs Itroducto Mture Destes ad Idetfablty ML Estmates Applcato to Normal Mtures Other Neural Networs Itroducto Prevously, all
More informationLecture 3. Sampling, sampling distributions, and parameter estimation
Lecture 3 Samplg, samplg dstrbutos, ad parameter estmato Samplg Defto Populato s defed as the collecto of all the possble observatos of terest. The collecto of observatos we take from the populato s called
More informationLecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model
Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The
More informationAssignment 5/MATH 247/Winter Due: Friday, February 19 in class (!) (answers will be posted right after class)
Assgmet 5/MATH 7/Wter 00 Due: Frday, February 9 class (!) (aswers wll be posted rght after class) As usual, there are peces of text, before the questos [], [], themselves. Recall: For the quadratc form
More informationRegression and the LMS Algorithm
CSE 556: Itroducto to Neural Netorks Regresso ad the LMS Algorthm CSE 556: Regresso 1 Problem statemet CSE 556: Regresso Lear regresso th oe varable Gve a set of N pars of data {, d }, appromate d b a
More information12.2 Estimating Model parameters Assumptions: ox and y are related according to the simple linear regression model
1. Estmatg Model parameters Assumptos: ox ad y are related accordg to the smple lear regresso model (The lear regresso model s the model that says that x ad y are related a lear fasho, but the observed
More informationSupport vector machines
CS 75 Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Outle Outle: Algorthms for lear decso boudary Support vector maches Mamum marg hyperplae.
More informationMULTIDIMENSIONAL HETEROGENEOUS VARIABLE PREDICTION BASED ON EXPERTS STATEMENTS. Gennadiy Lbov, Maxim Gerasimov
Iteratoal Boo Seres "Iformato Scece ad Computg" 97 MULTIIMNSIONAL HTROGNOUS VARIABL PRICTION BAS ON PRTS STATMNTS Geady Lbov Maxm Gerasmov Abstract: I the wors [ ] we proposed a approach of formg a cosesus
More informationSimple Linear Regression
Statstcal Methods I (EST 75) Page 139 Smple Lear Regresso Smple regresso applcatos are used to ft a model descrbg a lear relatoshp betwee two varables. The aspects of least squares regresso ad correlato
More informationGenerative classification models
CS 75 Mache Learg Lecture Geeratve classfcato models Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Data: D { d, d,.., d} d, Classfcato represets a dscrete class value Goal: lear f : X Y Bar classfcato
More informationFeature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture)
CSE 546: Mache Learg Lecture 6 Feature Selecto: Part 2 Istructor: Sham Kakade Greedy Algorthms (cotued from the last lecture) There are varety of greedy algorthms ad umerous amg covetos for these algorthms.
More informationDimensionality reduction Feature selection
CS 750 Mache Learg Lecture 3 Dmesoalty reducto Feature selecto Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 750 Mache Learg Dmesoalty reducto. Motvato. Classfcato problem eample: We have a put data
More informationCS 1675 Introduction to Machine Learning Lecture 12 Support vector machines
CS 675 Itroducto to Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Mdterm eam October 9, 7 I-class eam Closed book Stud materal: Lecture otes Correspodg chapters
More informationMEASURES OF DISPERSION
MEASURES OF DISPERSION Measure of Cetral Tedecy: Measures of Cetral Tedecy ad Dsperso ) Mathematcal Average: a) Arthmetc mea (A.M.) b) Geometrc mea (G.M.) c) Harmoc mea (H.M.) ) Averages of Posto: a) Meda
More informationESS Line Fitting
ESS 5 014 17. Le Fttg A very commo problem data aalyss s lookg for relatoshpetwee dfferet parameters ad fttg les or surfaces to data. The smplest example s fttg a straght le ad we wll dscuss that here
More informationLecture 8: Linear Regression
Lecture 8: Lear egresso May 4, GENOME 56, Sprg Goals Develop basc cocepts of lear regresso from a probablstc framework Estmatg parameters ad hypothess testg wth lear models Lear regresso Su I Lee, CSE
More informationCorrelation and Regression Analysis
Chapter V Correlato ad Regresso Aalss R. 5.. So far we have cosdered ol uvarate dstrbutos. Ma a tme, however, we come across problems whch volve two or more varables. Ths wll be the subject matter of the
More informationSummary of the lecture in Biostatistics
Summary of the lecture Bostatstcs Probablty Desty Fucto For a cotuos radom varable, a probablty desty fucto s a fucto such that: 0 dx a b) b a dx A probablty desty fucto provdes a smple descrpto of the
More informationCHAPTER 4 RADICAL EXPRESSIONS
6 CHAPTER RADICAL EXPRESSIONS. The th Root of a Real Number A real umber a s called the th root of a real umber b f Thus, for example: s a square root of sce. s also a square root of sce ( ). s a cube
More informationA conic cutting surface method for linear-quadraticsemidefinite
A coc cuttg surface method for lear-quadratcsemdefte programmg Mohammad R. Osoorouch Calfora State Uversty Sa Marcos Sa Marcos, CA Jot wor wth Joh E. Mtchell RPI July 3, 2008 Outle: Secod-order coe: defto
More informationSolving Constrained Flow-Shop Scheduling. Problems with Three Machines
It J Cotemp Math Sceces, Vol 5, 2010, o 19, 921-929 Solvg Costraed Flow-Shop Schedulg Problems wth Three Maches P Pada ad P Rajedra Departmet of Mathematcs, School of Advaced Sceces, VIT Uversty, Vellore-632
More informationObjectives of Multiple Regression
Obectves of Multple Regresso Establsh the lear equato that best predcts values of a depedet varable Y usg more tha oe eplaator varable from a large set of potetal predctors {,,... k }. Fd that subset of
More informationChapter Business Statistics: A First Course Fifth Edition. Learning Objectives. Correlation vs. Regression. In this chapter, you learn:
Chapter 3 3- Busess Statstcs: A Frst Course Ffth Edto Chapter 2 Correlato ad Smple Lear Regresso Busess Statstcs: A Frst Course, 5e 29 Pretce-Hall, Ic. Chap 2- Learg Objectves I ths chapter, you lear:
More informationCan we take the Mysticism Out of the Pearson Coefficient of Linear Correlation?
Ca we tae the Mstcsm Out of the Pearso Coeffcet of Lear Correlato? Itroducto As the ttle of ths tutoral dcates, our purpose s to egeder a clear uderstadg of the Pearso coeffcet of lear correlato studets
More informationCS 2750 Machine Learning. Lecture 8. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x
CS 75 Mache Learg Lecture 8 Lear regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Lear regresso Fucto f : X Y s a lear combato of put compoets f + + + K d d K k - parameters
More informationCOMPROMISE HYPERSPHERE FOR STOCHASTIC DOMINANCE MODEL
Sebasta Starz COMPROMISE HYPERSPHERE FOR STOCHASTIC DOMINANCE MODEL Abstract The am of the work s to preset a method of rakg a fte set of dscrete radom varables. The proposed method s based o two approaches:
More informationA tighter lower bound on the circuit size of the hardest Boolean functions
Electroc Colloquum o Computatoal Complexty, Report No. 86 2011) A tghter lower boud o the crcut sze of the hardest Boolea fuctos Masak Yamamoto Abstract I [IPL2005], Fradse ad Mlterse mproved bouds o the
More informationThe Selection Problem - Variable Size Decrease/Conquer (Practice with algorithm analysis)
We have covered: Selecto, Iserto, Mergesort, Bubblesort, Heapsort Next: Selecto the Qucksort The Selecto Problem - Varable Sze Decrease/Coquer (Practce wth algorthm aalyss) Cosder the problem of fdg the
More informationAbout a Fuzzy Distance between Two Fuzzy Partitions and Application in Attribute Reduction Problem
BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND IORMATION TECHNOLOGIES Volume 6, No 4 Sofa 206 Prt ISSN: 3-9702; Ole ISSN: 34-408 DOI: 0.55/cat-206-0064 About a Fuzzy Dstace betwee Two Fuzzy Parttos ad Applcato
More informationKernel-based Methods and Support Vector Machines
Kerel-based Methods ad Support Vector Maches Larr Holder CptS 570 Mache Learg School of Electrcal Egeerg ad Computer Scece Washgto State Uverst Refereces Muller et al. A Itroducto to Kerel-Based Learg
More informationECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity
ECONOMETRIC THEORY MODULE VIII Lecture - 6 Heteroskedastcty Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur . Breusch Paga test Ths test ca be appled whe the replcated data
More informationChapter 14 Logistic Regression Models
Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as
More informationLecture Notes Types of economic variables
Lecture Notes 3 1. Types of ecoomc varables () Cotuous varable takes o a cotuum the sample space, such as all pots o a le or all real umbers Example: GDP, Polluto cocetrato, etc. () Dscrete varables fte
More informationMachine Learning. Introduction to Regression. Le Song. CSE6740/CS7641/ISYE6740, Fall 2012
Mache Learg CSE6740/CS764/ISYE6740, Fall 0 Itroducto to Regresso Le Sog Lecture 4, August 30, 0 Based o sldes from Erc g, CMU Readg: Chap. 3, CB Mache learg for apartmet hutg Suppose ou are to move to
More informationMultiple Choice Test. Chapter Adequacy of Models for Regression
Multple Choce Test Chapter 06.0 Adequac of Models for Regresso. For a lear regresso model to be cosdered adequate, the percetage of scaled resduals that eed to be the rage [-,] s greater tha or equal to
More informationCLASS NOTES. for. PBAF 528: Quantitative Methods II SPRING Instructor: Jean Swanson. Daniel J. Evans School of Public Affairs
CLASS NOTES for PBAF 58: Quattatve Methods II SPRING 005 Istructor: Jea Swaso Dael J. Evas School of Publc Affars Uversty of Washgto Ackowledgemet: The structor wshes to thak Rachel Klet, Assstat Professor,
More informationBinary classification: Support Vector Machines
CS 57 Itroducto to AI Lecture 6 Bar classfcato: Support Vector Maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Supervsed learg Data: D { D, D,.., D} a set of eamples D, (,,,,,
More informationChapter 9 Jordan Block Matrices
Chapter 9 Jorda Block atrces I ths chapter we wll solve the followg problem. Gve a lear operator T fd a bass R of F such that the matrx R (T) s as smple as possble. f course smple s a matter of taste.
More informationL5 Polynomial / Spline Curves
L5 Polyomal / Sple Curves Cotets Coc sectos Polyomal Curves Hermte Curves Bezer Curves B-Sples No-Uform Ratoal B-Sples (NURBS) Mapulato ad Represetato of Curves Types of Curve Equatos Implct: Descrbe a
More informationFunctions of Random Variables
Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,
More informationIdeal multigrades with trigonometric coefficients
Ideal multgrades wth trgoometrc coeffcets Zarathustra Brady December 13, 010 1 The problem A (, k) multgrade s defed as a par of dstct sets of tegers such that (a 1,..., a ; b 1,..., b ) a j = =1 for all
More informationEvaluation of uncertainty in measurements
Evaluato of ucertaty measuremets Laboratory of Physcs I Faculty of Physcs Warsaw Uversty of Techology Warszawa, 05 Itroducto The am of the measuremet s to determe the measured value. Thus, the measuremet
More informationPart 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971))
art 4b Asymptotc Results for MRR usg RESS Recall that the RESS statstc s a specal type of cross valdato procedure (see Alle (97)) partcular to the regresso problem ad volves fdg Y $,, the estmate at the
More informationBeam Warming Second-Order Upwind Method
Beam Warmg Secod-Order Upwd Method Petr Valeta Jauary 6, 015 Ths documet s a part of the assessmet work for the subject 1DRP Dfferetal Equatos o Computer lectured o FNSPE CTU Prague. Abstract Ths documet
More informationCIS 800/002 The Algorithmic Foundations of Data Privacy October 13, Lecture 9. Database Update Algorithms: Multiplicative Weights
CIS 800/002 The Algorthmc Foudatos of Data Prvacy October 13, 2011 Lecturer: Aaro Roth Lecture 9 Scrbe: Aaro Roth Database Update Algorthms: Multplcatve Weghts We ll recall aga) some deftos from last tme:
More informationLecture 5: Interpolation. Polynomial interpolation Rational approximation
Lecture 5: Iterpolato olyomal terpolato Ratoal appromato Coeffcets of the polyomal Iterpolato: Sometme we kow the values of a fucto f for a fte set of pots. Yet we wat to evaluate f for other values perhaps
More informationMean is only appropriate for interval or ratio scales, not ordinal or nominal.
Mea Same as ordary average Sum all the data values ad dvde by the sample sze. x = ( x + x +... + x Usg summato otato, we wrte ths as x = x = x = = ) x Mea s oly approprate for terval or rato scales, ot
More informationFor combinatorial problems we might need to generate all permutations, combinations, or subsets of a set.
Addtoal Decrease ad Coquer Algorthms For combatoral problems we mght eed to geerate all permutatos, combatos, or subsets of a set. Geeratg Permutatos If we have a set f elemets: { a 1, a 2, a 3, a } the
More informationPTAS for Bin-Packing
CS 663: Patter Matchg Algorthms Scrbe: Che Jag /9/00. Itroducto PTAS for B-Packg The B-Packg problem s NP-hard. If we use approxmato algorthms, the B-Packg problem could be solved polyomal tme. For example,
More informationABOUT ONE APPROACH TO APPROXIMATION OF CONTINUOUS FUNCTION BY THREE-LAYERED NEURAL NETWORK
ABOUT ONE APPROACH TO APPROXIMATION OF CONTINUOUS FUNCTION BY THREE-LAYERED NEURAL NETWORK Ram Rzayev Cyberetc Isttute of the Natoal Scece Academy of Azerbaa Republc ramrza@yahoo.com Aygu Alasgarova Khazar
More informationRegresso What s a Model? 1. Ofte Descrbe Relatoshp betwee Varables 2. Types - Determstc Models (o radomess) - Probablstc Models (wth radomess) EPI 809/Sprg 2008 9 Determstc Models 1. Hypothesze
More informationf f... f 1 n n (ii) Median : It is the value of the middle-most observation(s).
CHAPTER STATISTICS Pots to Remember :. Facts or fgures, collected wth a defte pupose, are called Data.. Statstcs s the area of study dealg wth the collecto, presetato, aalyss ad terpretato of data.. The
More information( ) 2 2. Multi-Layer Refraction Problem Rafael Espericueta, Bakersfield College, November, 2006
Mult-Layer Refracto Problem Rafael Espercueta, Bakersfeld College, November, 006 Lght travels at dfferet speeds through dfferet meda, but refracts at layer boudares order to traverse the least-tme path.
More informationGeneralized Linear Regression with Regularization
Geeralze Lear Regresso wth Regularzato Zoya Bylsk March 3, 05 BASIC REGRESSION PROBLEM Note: I the followg otes I wll make explct what s a vector a what s a scalar usg vec t or otato, to avo cofuso betwee
More informationECON 5360 Class Notes GMM
ECON 560 Class Notes GMM Geeralzed Method of Momets (GMM) I beg by outlg the classcal method of momets techque (Fsher, 95) ad the proceed to geeralzed method of momets (Hase, 98).. radtoal Method of Momets
More informationLecture 16: Backpropogation Algorithm Neural Networks with smooth activation functions
CO-511: Learg Theory prg 2017 Lecturer: Ro Lv Lecture 16: Bacpropogato Algorthm Dsclamer: These otes have ot bee subected to the usual scruty reserved for formal publcatos. They may be dstrbuted outsde
More informationLINEAR REGRESSION ANALYSIS
LINEAR REGRESSION ANALYSIS MODULE V Lecture - Correctg Model Iadequaces Through Trasformato ad Weghtg Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur Aalytcal methods for
More informationOrdinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
More informationarxiv: v1 [cs.lg] 22 Feb 2015
SDCA wthout Dualty Sha Shalev-Shwartz arxv:50.0677v cs.lg Feb 05 Abstract Stochastc Dual Coordate Ascet s a popular method for solvg regularzed loss mmzato for the case of covex losses. I ths paper we
More informationEECE 301 Signals & Systems
EECE 01 Sgals & Systems Prof. Mark Fowler Note Set #9 Computg D-T Covoluto Readg Assgmet: Secto. of Kame ad Heck 1/ Course Flow Dagram The arrows here show coceptual flow betwee deas. Note the parallel
More informationIntroduction to local (nonparametric) density estimation. methods
Itroducto to local (oparametrc) desty estmato methods A slecture by Yu Lu for ECE 66 Sprg 014 1. Itroducto Ths slecture troduces two local desty estmato methods whch are Parze desty estmato ad k-earest
More informationENGI 3423 Simple Linear Regression Page 12-01
ENGI 343 mple Lear Regresso Page - mple Lear Regresso ometmes a expermet s set up where the expermeter has cotrol over the values of oe or more varables X ad measures the resultg values of aother varable
More informationAN UPPER BOUND FOR THE PERMANENT VERSUS DETERMINANT PROBLEM BRUNO GRENET
AN UPPER BOUND FOR THE PERMANENT VERSUS DETERMINANT PROBLEM BRUNO GRENET Abstract. The Permaet versus Determat problem s the followg: Gve a matrx X of determates over a feld of characterstc dfferet from
More informationC. Statistics. X = n geometric the n th root of the product of numerical data ln X GM = or ln GM = X 2. X n X 1
C. Statstcs a. Descrbe the stages the desg of a clcal tral, takg to accout the: research questos ad hypothess, lterature revew, statstcal advce, choce of study protocol, ethcal ssues, data collecto ad
More informationMA/CSSE 473 Day 27. Dynamic programming
MA/CSSE 473 Day 7 Dyamc Programmg Bomal Coeffcets Warshall's algorthm (Optmal BSTs) Studet questos? Dyamc programmg Used for problems wth recursve solutos ad overlappg subproblems Typcally, we save (memoze)
More informationSupport vector machines II
CS 75 Mache Learg Lecture Support vector maches II Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Learl separable classes Learl separable classes: here s a hperplae that separates trag staces th o error
More informationMaximum Likelihood Estimation
Marquette Uverst Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Coprght 08 b Marquette Uverst Maxmum Lkelhood Estmato We have bee sag that ~
More informationInvestigation of Partially Conditional RP Model with Response Error. Ed Stanek
Partally Codtoal Radom Permutato Model 7- vestgato of Partally Codtoal RP Model wth Respose Error TRODUCTO Ed Staek We explore the predctor that wll result a smple radom sample wth respose error whe a
More information8.1 Hashing Algorithms
CS787: Advaced Algorthms Scrbe: Mayak Maheshwar, Chrs Hrchs Lecturer: Shuch Chawla Topc: Hashg ad NP-Completeess Date: September 21 2007 Prevously we looked at applcatos of radomzed algorthms, ad bega
More informationLecture 12 APPROXIMATION OF FIRST ORDER DERIVATIVES
FDM: Appromato of Frst Order Dervatves Lecture APPROXIMATION OF FIRST ORDER DERIVATIVES. INTRODUCTION Covectve term coservato equatos volve frst order dervatves. The smplest possble approach for dscretzato
More informationBlock-Based Compact Thermal Modeling of Semiconductor Integrated Circuits
Block-Based Compact hermal Modelg of Semcoductor Itegrated Crcuts Master s hess Defese Caddate: Jg Ba Commttee Members: Dr. Mg-Cheg Cheg Dr. Daqg Hou Dr. Robert Schllg July 27, 2009 Outle Itroducto Backgroud
More information: At least two means differ SST
Formula Card for Eam 3 STA33 ANOVA F-Test: Completely Radomzed Desg ( total umber of observatos, k = Number of treatmets,& T = total for treatmet ) Step : Epress the Clam Step : The ypotheses: :... 0 A
More informationUnimodality Tests for Global Optimization of Single Variable Functions Using Statistical Methods
Malaysa Umodalty Joural Tests of Mathematcal for Global Optmzato Sceces (): of 05 Sgle - 5 Varable (007) Fuctos Usg Statstcal Methods Umodalty Tests for Global Optmzato of Sgle Varable Fuctos Usg Statstcal
More informationLecture Notes 2. The ability to manipulate matrices is critical in economics.
Lecture Notes. Revew of Matrces he ablt to mapulate matrces s crtcal ecoomcs.. Matr a rectagular arra of umbers, parameters, or varables placed rows ad colums. Matrces are assocated wth lear equatos. lemets
More information6.867 Machine Learning
6.867 Mache Learg Problem set Due Frday, September 9, rectato Please address all questos ad commets about ths problem set to 6.867-staff@a.mt.edu. You do ot eed to use MATLAB for ths problem set though
More informationSTK4011 and STK9011 Autumn 2016
STK4 ad STK9 Autum 6 Pot estmato Covers (most of the followg materal from chapter 7: Secto 7.: pages 3-3 Secto 7..: pages 3-33 Secto 7..: pages 35-3 Secto 7..3: pages 34-35 Secto 7.3.: pages 33-33 Secto
More information{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:
Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed
More informationChapter Statistics Background of Regression Analysis
Chapter 06.0 Statstcs Backgroud of Regresso Aalyss After readg ths chapter, you should be able to:. revew the statstcs backgroud eeded for learg regresso, ad. kow a bref hstory of regresso. Revew of Statstcal
More informationUNIT 2 SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS
Numercal Computg -I UNIT SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS Structure Page Nos..0 Itroducto 6. Objectves 7. Ital Approxmato to a Root 7. Bsecto Method 8.. Error Aalyss 9.4 Regula Fals Method
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Postpoed exam: ECON430 Statstcs Date of exam: Jauary 0, 0 Tme for exam: 09:00 a.m. :00 oo The problem set covers 5 pages Resources allowed: All wrtte ad prted
More informationTraining Sample Model: Given n observations, [[( Yi, x i the sample model can be expressed as (1) where, zero and variance σ
Stat 74 Estmato for Geeral Lear Model Prof. Goel Broad Outle Geeral Lear Model (GLM): Trag Samle Model: Gve observatos, [[( Y, x ), x = ( x,, xr )], =,,, the samle model ca be exressed as Y = µ ( x, x,,
More informationPoint Estimation: definition of estimators
Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.
More informationMidterm Exam 1, section 1 (Solution) Thursday, February hour, 15 minutes
coometrcs, CON Sa Fracsco State Uversty Mchael Bar Sprg 5 Mdterm am, secto Soluto Thursday, February 6 hour, 5 mutes Name: Istructos. Ths s closed book, closed otes eam.. No calculators of ay kd are allowed..
More informationSimple Linear Regression
Correlato ad Smple Lear Regresso Berl Che Departmet of Computer Scece & Iformato Egeerg Natoal Tawa Normal Uversty Referece:. W. Navd. Statstcs for Egeerg ad Scetsts. Chapter 7 (7.-7.3) & Teachg Materal
More informationLecture 07: Poles and Zeros
Lecture 07: Poles ad Zeros Defto of poles ad zeros The trasfer fucto provdes a bass for determg mportat system respose characterstcs wthout solvg the complete dfferetal equato. As defed, the trasfer fucto
More informationThe equation is sometimes presented in form Y = a + b x. This is reasonable, but it s not the notation we use.
INTRODUCTORY NOTE ON LINEAR REGREION We have data of the form (x y ) (x y ) (x y ) These wll most ofte be preseted to us as two colum of a spreadsheet As the topc develops we wll see both upper case ad
More informationChapter 13 Student Lecture Notes 13-1
Chapter 3 Studet Lecture Notes 3- Basc Busess Statstcs (9 th Edto) Chapter 3 Smple Lear Regresso 4 Pretce-Hall, Ic. Chap 3- Chapter Topcs Types of Regresso Models Determg the Smple Lear Regresso Equato
More informationThird handout: On the Gini Index
Thrd hadout: O the dex Corrado, a tala statstca, proposed (, 9, 96) to measure absolute equalt va the mea dfferece whch s defed as ( / ) where refers to the total umber of dvduals socet. Assume that. The
More informationMultivariate Transformation of Variables and Maximum Likelihood Estimation
Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty
More informationEconometric Methods. Review of Estimation
Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators
More informationPrincipal Components. Analysis. Basic Intuition. A Method of Self Organized Learning
Prcpal Compoets Aalss A Method of Self Orgazed Learg Prcpal Compoets Aalss Stadard techque for data reducto statstcal patter matchg ad sgal processg Usupervsed learg: lear from examples wthout a teacher
More information1 Onto functions and bijections Applications to Counting
1 Oto fuctos ad bectos Applcatos to Coutg Now we move o to a ew topc. Defto 1.1 (Surecto. A fucto f : A B s sad to be surectve or oto f for each b B there s some a A so that f(a B. What are examples of
More informationLecture 1: Introduction to Regression
Lecture : Itroducto to Regresso A Eample: Eplag State Homcde Rates What kds of varables mght we use to epla/predct state homcde rates? Let s cosder just oe predctor for ow: povert Igore omtted varables,
More information