Boosting and Ensemble Methods
|
|
- Holly Reeves
- 5 years ago
- Views:
Transcription
1 Boosng and Ensmbl Mhods
2 PAC Larnng modl Som dsrbuon D ovr doman X Eampls: <, c*> c* s h arg funcon Goal: Wh hgh probably -d fnd h n H such ha rrorh,c* < d and ar arbrarly small. Inro o ML 2
3 Wak Larnng - Confdnc Assum w ar gvn a larnng algorhm wh confdnc d ½ bu arbrary small accuracy ε>0 Can w boos h confdnc? How?! Inro o ML 3
4 BoosConfdnc Algorhm Inpu: Algorhm A and paramr d Cra klog2/d ndpndn problms Sampl S for -h copy Run Algorhm A on S wh accuracy /3 L h b h hypohss ha A oupus on S Tak a nw sampl S of sz m -2 log2k/d Rurn h bs hypohss h on S call h* Inro o ML 4
5 BoosConfdnc Analyss S For ach copy : Th probably ha rror h < /3 a las ½ Th probably ha som h has rrorh < /3 a las -2 -k - d/2 Holds for klog2/d Assum hs holds! Namly, som h has rrorh < /3 dno by h Inro o ML 5 +
6 BoosConfdnc Analyss Wh probably a las - d/2 for vry h : rrorh - [obs-rrorh ] ε/3 Chrnoff bound + unon bound usng m -2 log2k/d Assum hs holds! Toghr, wh probably a las -d rrorh* Errorh + ε/3 Obs-Errorh + 2ε/3 Obs-Errorh * Obs-Errorh + 2ε/3 Errorh * Obs_Error h * + ε/3 ε Inro o ML 6
7 Wak Larnng - Accuracy Assum: rrorh,c* < ½-g Th paramr g>0 s small consan Inuvly: gng slghly br han random should b asy Quson: Assum C s wak larnabl, s C PAC srong larnabl? Inro o ML 7
8 Wak Larnng - Dfnon Wak Larnng Algo A wak larns C usng H f ss γ>0 for all c n C for any dsrbuon D for all δ> 0 Oupus h n H such ha wh prob - δ rrorh,c < ½ - γ Srong Larnng Algo A srong larns C usng H f for all ε>0 for all c n C for any dsrbuon D for all δ> 0 Oupus h n H such ha wh prob - δ rrorh,c < ε Inro o ML 8
9 Wak larnng dfnon Why do w nd ANY dsrbuon Eampl: Consdr h followng dsrbuon ovr bs f 2 0 hn c* hard funcon ohrws c*0 Unform Dsrbuon Prdcng random for 2 0, 0 lswhr wll b corrc 87.5% Gng abov ha s mpossbl. Th hard dsrbuon: Sampl from h s whr 2 0 Inro o ML 9
10 Thr Wak Larnrs On wak larnr only on hng o do! Two wak larnrs wha o do f hy dsagr? Thr wak larnrs Can w mprov accuracy? Eampl - y - y Wak larnrs: sngl bs Inro o ML 0
11 Thr wak larnrs Frs wak larnr us h dsrbuon D g h Scond wak larnr How can w forc nw h? S D 2 s.. h has rror ½ G h 2 why h 2 h? Thrd wak larnr Wha ar nrsng npus? h 2 h L D 3 b such npus g h 3 How wll w prdc? If h 2 h usng h 3 Els h 2 or h majory Inro o ML
12 3 Wak Larnrs - Prformanc Dfn D 2 and D 3 Ds D 2 : Slc random b{0,} If b0 Sampl from D unl h c* Els b Sampl from D unl h c* Ds D 3 Sampl from D unl h h 2 Inro o ML Formally 0.5 D f - p D2 0.5 D f p p Pr[h c* ] D D3 Z 0 Z Pr[h h 2 f f ] h h h h c* h c* 2 h 2 2
13 3 Wak Larnrs - Analyss Assum, for smplcy, ha all WL hav rror ra p½ -γ If all rrors ar ndpndn, h rror of h majory s Error 3p 2 -p+p 3 3p 2-2p 3 Goal: holds vn for dpndn rrors Inro o ML 3
14 3 Wak Larnrs - Analyss Assum all WL ar p½ -γ To convr from a probably n D 2 o D 0.5 D f - p D2 0.5 D f p p Pr[h c* ] h h c* c* Corrc n h : 2-p Error n h : 2p Inro o ML 4
15 c: 2-p : 2p 3 Wak Larnrs - Analyss Assum all WL ar p½ -γ P c P cc P c P h on D h on D 2 h 2 on D 2 Inro o ML 5
16 c: 2-p : 2p 3 Wak Larnrs - Analyss Assum all WL ar p½ -γ P c P cc P c P h on D h on D 2 h 2 on D 2 a P * P c +P Inro o ML p-a 6
17 c: 2-p : 2p 3 Wak Larnrs - Analyss Assum all WL ar p½ -γ P c P cc P c P h on D h on D 2 h 2 on D 2 a ½-p+a Inro o ML ½ p-a 7
18 c: 2-p : 2p 3 Wak Larnrs - Analyss Assum all WL ar p½ -γ P c P cc P c P 2-pa 2p½-p+a 2pp-a h on D h 2 on D 2 a ½-p+a Inro o ML ½ p-a 8
19 3 Wak Larnrs - Analyss Assum all WL ar p½ -γ P c P cc P c P h on D 2-pa 2p½-p+a 2pp-a Error P +pp c +P c 3p 2-2p 3 Inro o ML 9
20 3 Wak Larnrs - Analyss Assum all WL ar p½ -γ P c P cc P c P h on D 2-pa 2p½-p+a 2pp-a Error P +pp c +P c 3p 2-2p 3 p3p-2p 2 p-4γ 2 Inro o ML 20
21 3 Wak Larnrs - Analyss Nw Error Old Error*-4γ 2-4γ 2 Inro o ML γ 2
22 Wha abou mor hypohss Th CS way Do rcursvly Can push down h rror arbrarly W show a mor consrucv way Inro o ML 22
23 ADABOOST: ADAPTIVE BOOSTING Inro o ML 23
24 AdaBoos: Ovrvw Buld a lnar classfr basc lmns, wak larnrs An onln approach ach m add on mor classfr F h sampl S Each m sp hav a hypohss f Slc a dsrbuon D on S Fnd a wak larnr h w.r. D Add h o h hypohss dcd on wgh a f + f +a h prdc sgnf + Inro o ML 24
25 AdaBoos: Algorhm Wak Larnrs H h: X{+,-} Inalzaon: FIXED Sampl S{,y..., m,y m } D /m Prdcon fsgnσα h Sp,, T D Inro o ML Rcv h WL w.r.. D Dfn ε Pr[h c*] α ½ log - ε / ε Dfn D +, Z D Z D - - y h y y h 26 h
26 AdaBoos: Inuon How do w chang h dsrbuon? Error wgh ncrass Corrc wgh dcrass Focus on h hard ampls Wha ar h paramrs? Th wak larnng class H Th numbr of raons T Assum o b npus Inro o ML 27
27 Illusrang AdaBoos Inal wghs for ach daa pon Orgnal Daa B Boosng Round
28 Boosng Illusrang AdaBoos B Round Boosng B Round B Boosng Round Ovrall
29 AdaBoos: Analyss Thorm: Gvn,..., T h ranng rror of f s boundd by T T 2 - Proof basd on hr clams Inro o ML 30
30 AdaBoos: Analyss Clam : whr fσα h Corollary : Proof For + w hav unravl rcurrnc - f y T Z D D h y Z D D f y h y T h y T Z D Z D Z D D T - T f y Z md 3 m D / snc Inro o ML
31 AdaBoos: Analyss Clam 2: Proof T Z S f rror, - m T T T m f y m m Z D Z Z md m m f y I m f sgn y I m S f rror 0, Corollary z<0 -z > 32 Inro o ML
32 AdaBoos: Analyss Clam 3: Proof 2 Z - D D D Z h y h y h y m : : rcall : h y D 33 α ½ log - ε / ε and subsu Inro o ML α ε ε
33 AdaBoos: Analyss Why hs valu of alpha? vald for any α Mnmz Z o rduc rror clam 2 2 ln 2 2 Z d dz Z - - Inro o ML
34 AdaBoos: Analyss Thorm: Gvn,..., T h rror of f s boundd by T T T g g 35 T Z Inro o ML
35 AdaBoos: Analyss Thorm: Gvn,..., T h rror of f s boundd by For 2 T T g > - g 2Tg - 2 T Inro o ML T - 4g 2 Z -2 g 2 36
36 Ensmbl Mhods
37 Ensmbl Mhods Hgh lvl da Gnra mulpl hypohss Combn hm o a sngl classfr Two mporan qusons How do w gnra mulpl hypohss w hav only on sampl How do w combn h mulpl hypohss Majory, AdaBoos, Inro o ML 39
38 Raonal for Ensmbl Mhods Sascal Compuaonal h f h h h h h f h h h f h Rprsnaonal Inro o ML 40 Sourc: hp://wb.ngr.orgonsa.du/~gd/publcaons/mcs-nsmbls.pdf
39 Boosng Boosng s acually an nsmbl mhod Gnrang dffrn hypohss: By changng h sampl dsrbuon Combnng hypohss wghd lnar prdcor Wghs drmn whn hypo. s slcd. Inro o ML 4
40 Baggng Inpu: a sngl larnng algorhm A How do w gnra dffrn Hypohss samplng wh rplacmn manans h sascs Formally, gvn a sampl S Sub sampl S,, S k Run A on S o gnra h Combnng: Smpl majory Inro o ML 42
41 Bas vs. Varanc Man Squar Error MSE E h [ f-h 2 ] Bas 2 + Var Whr Bas E[f-h] Var E[h 2 ] E 2 [h] MSE var bas 2 Inro o ML 43
42 Baggng raonal: Bas vrsus Var Why s on hypohss wors han many?! Epcd rror of h dncal o all h wors han ranng on all sampl smallr sampl BIAS Varanc of h rror sngl hypohss - flucuas consdrably majory of many - much mor sabl Mor sabl br gnralzaon h ranng rror br rflcs h ru rror Inro o ML 44
43 Dcson Tr vs. Baggng Sourc: hp:// Inro o ML 45
44 Sackng Inpu: Sampl S k algorhms A combng algo C Run A on S gnra h Gvn h,, h k gnra nw sampl,y h,,h k,y Run C o gnra ħ Oupu ħ Wha can b A? Wha can b C? Baggng: A sub-sampls C s a majory AdaBoos: A wak hypo m C wghd majory Inro o ML 46
45 Random Fors: movaon Dcson Trs: Bas Dcson r craon s vry nosy Dpnds on parcular sampl Lowrng Varanc: Avragng ovr dcson rs How can w gnra dffrn dcson rs? Sub-sampl h sampl Forc cran arbus Inro o ML 47
46 Random Fors: Algorhm Cra K dffrn dcson rs: Sampl: Slc a random sub-sampl Pracc: 66% GOAL: Gnra a vary of DT Wll corrlad wh y Combnng: Majory Arbus: In ach nod slc subs F of arbus F M Wak larnrs Slc h bs ar. n F Valus of M: M: random MN all arbus rgular DT << M<< N Subs of arbus Popular: M N Inro o ML 48
47 Random Fors Sourc: hp://cs.sanford.du/popl/karpahy/randomforsspral.png Inro o ML 49
48 Random Fors: Concluson Bnfs: Fas o run Farly sabl oucom Compv prformanc Handls mssng/paral daa Waknsss: Losss nrprably unlk DT Many paramrs Bu sms robus Faur slcon Collds wh arbus samplng Inro o ML 50
49 MNIST: Comparav rsuls Classfr Accuracy Tranng Tm Tsng Tm Nural N 97.80% s 0.398s Lnar SVM 94.6% s 58.00s Random Fors 96.4% 2.359s s k nars nghbors k % s s Dcson Tr dph % 3.346s 0.033s Adaboos 73.67% s.585s Sourc: hps://marn-homa.com/comparng-classfrs/ Inro o ML 5
50 Concluson Boosng Wak vs srong larnng Thory 3 o Pracc AdaBoos Ensmbl Mhods Boosng Baggng Sackng Random Fors Inro o ML 52
Consider a system of 2 simultaneous first order linear equations
Soluon of sysms of frs ordr lnar quaons onsdr a sysm of smulanous frs ordr lnar quaons a b c d I has h alrna mar-vcor rprsnaon a b c d Or, n shorhand A, f A s alrady known from con W know ha h abov sysm
More informationThe Variance-Covariance Matrix
Th Varanc-Covaranc Marx Our bggs a so-ar has bn ng a lnar uncon o a s o daa by mnmzng h las squars drncs rom h o h daa wh mnsarch. Whn analyzng non-lnar daa you hav o us a program l Malab as many yps o
More informationLecture 4 : Backpropagation Algorithm. Prof. Seul Jung ( Intelligent Systems and Emotional Engineering Laboratory) Chungnam National University
Lcur 4 : Bacpropagaon Algorhm Pro. Sul Jung Inllgn Sm and moonal ngnrng Laboraor Chungnam Naonal Unvr Inroducon o Bacpropagaon algorhm 969 Mn and Papr aac. 980 Parr and Wrbo dcovrd bac propagaon algorhm.
More information9. Simple Rules for Monetary Policy
9. Smpl Ruls for Monar Polc John B. Talor, Ma 0, 03 Woodford, AR 00 ovrvw papr Purpos s o consdr o wha xn hs prscrpon rsmbls h sor of polc ha conomc hor would rcommnd Bu frs, l s rvw how hs sor of polc
More informationSummary: Solving a Homogeneous System of Two Linear First Order Equations in Two Unknowns
Summary: Solvng a Homognous Sysm of Two Lnar Frs Ordr Equaons n Two Unknowns Gvn: A Frs fnd h wo gnvalus, r, and hr rspcv corrspondng gnvcors, k, of h coffcn mar A Dpndng on h gnvalus and gnvcors, h gnral
More informationt=0 t>0: + vr - i dvc Continuation
hapr Ga Dlay and rcus onnuaon s rcu Equaon >: S S Ths dffrnal quaon, oghr wh h nal condon, fully spcfs bhaor of crcu afr swch closs Our n challng: larn how o sol such quaons TUE/EE 57 nwrk analys 4/5 NdM
More informationFrequency Response. Response of an LTI System to Eigenfunction
Frquncy Rsons Las m w Rvsd formal dfnons of lnary and m-nvaranc Found an gnfuncon for lnar m-nvaran sysms Found h frquncy rsons of a lnar sysm o gnfuncon nu Found h frquncy rsons for cascad, fdbac, dffrnc
More informationState Observer Design
Sa Obsrvr Dsgn A. Khak Sdgh Conrol Sysms Group Faculy of Elcrcal and Compur Engnrng K. N. Toos Unvrsy of Tchnology Fbruary 2009 1 Problm Formulaon A ky assumpon n gnvalu assgnmn and sablzng sysms usng
More informationAdvanced Queueing Theory. M/G/1 Queueing Systems
Advand Quung Thory Ths slds ar rad by Dr. Yh Huang of Gorg Mason Unvrsy. Sudns rgsrd n Dr. Huang's ourss a GMU an ma a sngl mahn-radabl opy and prn a sngl opy of ah sld for hr own rfrn, so long as ah sld
More informationIntroduction to Boosting
Inroducon o Boosng Cynha Rudn PACM, Prnceon Unversy Advsors Ingrd Daubeches and Rober Schapre Say you have a daabase of news arcles, +, +, -, -, +, +, -, -, +, +, -, -, +, +, -, + where arcles are labeled
More informationFAULT TOLERANT SYSTEMS
FAULT TOLERANT SYSTEMS hp://www.cs.umass.du/c/orn/faultolransysms ar 4 Analyss Mhods Chapr HW Faul Tolranc ar.4.1 Duplx Sysms Boh procssors xcu h sam as If oupus ar n agrmn - rsul s assumd o b corrc If
More informationCentral University of Finance and Economics, Beijing, China. *Corresponding author
016 Jon Inrnaonal Confrnc on Arfcal Inllgnc and Copur Engnrng (AICE 016) and Inrnaonal Confrnc on Nwork and Councaon Scury (NCS 016) ISBN: 978-1-60595-36-5 AdaBoos Arfcal Nural Nwork for Sock Mark Prdcng
More informationinnovations shocks white noise
Innovaons Tm-srs modls ar consrucd as lnar funcons of fundamnal forcasng rrors, also calld nnovaons or shocks Ths basc buldng blocks sasf var σ Srall uncorrlad Ths rrors ar calld wh nos In gnral, f ou
More informationELEN E4830 Digital Image Processing
ELEN E48 Dgal Imag Procssng Mrm Eamnaon Sprng Soluon Problm Quanzaon and Human Encodng r k u P u P u r r 6 6 6 6 5 6 4 8 8 4 P r 6 6 P r 4 8 8 6 8 4 r 8 4 8 4 7 8 r 6 6 6 6 P r 8 4 8 P r 6 6 8 5 P r /
More informationUNIT #5 EXPONENTIAL AND LOGARITHMIC FUNCTIONS
Answr Ky Nam: Da: UNIT # EXPONENTIAL AND LOGARITHMIC FUNCTIONS Par I Qusions. Th prssion is quivaln o () () 6 6 6. Th ponnial funcion y 6 could rwrin as y () y y 6 () y y (). Th prssion a is quivaln o
More informationChapter 13 Laplace Transform Analysis
Chapr aplac Tranorm naly Chapr : Ouln aplac ranorm aplac Tranorm -doman phaor analy: x X σ m co ω φ x X X m φ x aplac ranorm: [ o ] d o d < aplac Tranorm Thr condon Unlaral on-dd aplac ranorm: aplac ranorm
More informationWave Superposition Principle
Physcs 36: Was Lcur 5 /7/8 Wa Suroson Prncl I s qu a common suaon for wo or mor was o arr a h sam on n sac or o xs oghr along h sam drcon. W wll consdr oday sral moran cass of h combnd ffcs of wo or mor
More informationProbabilistic Reasoning; Graphical models
Ovrvw. robablsc Rasonng/Graphcal modls. Imporanc Samplng 3. Markov Chan Mon Carlo: Gbbs Samplng 4. Samplng n prsnc of Drmnsm 5. Rao-Blackwllsaon 6. AND/OR mporanc samplng Ovrvw. robablsc Rasonng/Graphcal
More informationFinal Exam : Solutions
Comp : Algorihm and Daa Srucur Final Exam : Soluion. Rcuriv Algorihm. (a) To bgin ind h mdian o {x, x,... x n }. Sinc vry numbr xcp on in h inrval [0, n] appar xacly onc in h li, w hav ha h mdian mu b
More informationCPSC 211 Data Structures & Implementations (c) Texas A&M University [ 259] B-Trees
CPSC 211 Daa Srucurs & Implmnaions (c) Txas A&M Univrsiy [ 259] B-Trs Th AVL r and rd-black r allowd som variaion in h lnghs of h diffrn roo-o-laf pahs. An alrnaiv ida is o mak sur ha all roo-o-laf pahs
More informationSupplementary Figure 1. Experiment and simulation with finite qudit. anharmonicity. (a), Experimental data taken after a 60 ns three-tone pulse.
Supplmnar Fgur. Eprmn and smulaon wh fn qud anharmonc. a, Eprmnal daa akn afr a 6 ns hr-on puls. b, Smulaon usng h amlonan. Supplmnar Fgur. Phagoran dnamcs n h m doman. a, Eprmnal daa. Th hr-on puls s
More informationy = 2xe x + x 2 e x at (0, 3). solution: Since y is implicitly related to x we have to use implicit differentiation: 3 6y = 0 y = 1 2 x ln(b) ln(b)
4. y = y = + 5. Find th quation of th tangnt lin for th function y = ( + ) 3 whn = 0. solution: First not that whn = 0, y = (1 + 1) 3 = 8, so th lin gos through (0, 8) and thrfor its y-intrcpt is 8. y
More informationLucas Test is based on Euler s theorem which states that if n is any integer and a is coprime to n, then a φ(n) 1modn.
Modul 10 Addtonal Topcs 10.1 Lctur 1 Prambl: Dtrmnng whthr a gvn ntgr s prm or compost s known as prmalty tstng. Thr ar prmalty tsts whch mrly tll us whthr a gvn ntgr s prm or not, wthout gvng us th factors
More informationReview - Probabilistic Classification
Mmoral Unvrsty of wfoundland Pattrn Rcognton Lctur 8 May 5, 6 http://www.ngr.mun.ca/~charlsr Offc Hours: Tusdays Thursdays 8:3-9:3 PM E- (untl furthr notc) Gvn lablld sampls { ɛc,,,..., } {. Estmat Rvw
More informationHomework: Introduction to Motion
Homwork: Inroducon o Moon Dsanc vs. Tm Graphs Nam Prod Drcons: Answr h foowng qusons n h spacs provdd. 1. Wha do you do o cra a horzona n on a dsancm graph? 2. How do you wak o cra a sragh n ha sops up?
More informationA Note on Estimability in Linear Models
Intrnatonal Journal of Statstcs and Applcatons 2014, 4(4): 212-216 DOI: 10.5923/j.statstcs.20140404.06 A Not on Estmablty n Lnar Modls S. O. Adymo 1,*, F. N. Nwob 2 1 Dpartmnt of Mathmatcs and Statstcs,
More informationGrand Canonical Ensemble
Th nsmbl of systms mmrsd n a partcl-hat rsrvor at constant tmpratur T, prssur P, and chmcal potntal. Consdr an nsmbl of M dntcal systms (M =,, 3,...M).. Thy ar mutually sharng th total numbr of partcls
More informationDepartment of Economics University of Toronto
Deparmen of Economcs Unversy of Torono ECO408F M.A. Economercs Lecure Noes on Heeroskedascy Heeroskedascy o Ths lecure nvolves lookng a modfcaons we need o make o deal wh he regresson model when some of
More informationSIMEON BALL AND AART BLOKHUIS
A BOUND FOR THE MAXIMUM WEIGHT OF A LINEAR CODE SIMEON BALL AND AART BLOKHUIS Absrac. I s shown ha h paramrs of a lnar cod ovr F q of lngh n, dmnson k, mnmum wgh d and maxmum wgh m sasfy a cran congrunc
More informationLecture VI Regression
Lecure VI Regresson (Lnear Mehods for Regresson) Conens: Lnear Mehods for Regresson Leas Squares, Gauss Markov heorem Recursve Leas Squares Lecure VI: MLSC - Dr. Sehu Vjayakumar Lnear Regresson Model M
More informationCopyright 2000, Kevin Wayne 1
Rcap: Maxmum 3-Sasfably Maxmum 3-Sasfably: Analyss CS 580: Algorhm Dsgn and Analyss Jrmah Block Purdu Unvrsy Sprng 2018 Announcmns: Homwork 6 dadln xndd o Aprl 24 h a 11:59 PM Cours Evaluaon Survy: Lv
More informationAR(1) Process. The first-order autoregressive process, AR(1) is. where e t is WN(0, σ 2 )
AR() Procss Th firs-ordr auorgrssiv procss, AR() is whr is WN(0, σ ) Condiional Man and Varianc of AR() Condiional man: Condiional varianc: ) ( ) ( Ω Ω E E ) var( ) ) ( var( ) var( σ Ω Ω Ω Ω E Auocovarianc
More informationCHAPTER 2: Supervised Learning
HATER 2: Supervsed Learnng Learnng a lass from Eamples lass of a famly car redcon: Is car a famly car? Knowledge eracon: Wha do people epec from a famly car? Oupu: osve (+) and negave ( ) eamples Inpu
More informationCHAPTER 7d. DIFFERENTIATION AND INTEGRATION
CHAPTER 7d. DIFFERENTIATION AND INTEGRATION A. J. Clark School o Engnrng Dpartmnt o Cvl and Envronmntal Engnrng by Dr. Ibrahm A. Assakka Sprng ENCE - Computaton Mthods n Cvl Engnrng II Dpartmnt o Cvl and
More informationCHAPTER 10: LINEAR DISCRIMINATION
CHAPER : LINEAR DISCRIMINAION Dscrmnan-based Classfcaon 3 In classfcaon h K classes (C,C,, C k ) We defned dscrmnan funcon g j (), j=,,,k hen gven an es eample, e chose (predced) s class label as C f g
More informationCSE 245: Computer Aided Circuit Simulation and Verification
CSE 45: Compur Aidd Circui Simulaion and Vrificaion Fall 4, Sp 8 Lcur : Dynamic Linar Sysm Oulin Tim Domain Analysis Sa Equaions RLC Nwork Analysis by Taylor Expansion Impuls Rspons in im domain Frquncy
More informationLecture 6: Learning for Control (Generalised Linear Regression)
Lecure 6: Learnng for Conrol (Generalsed Lnear Regresson) Conens: Lnear Mehods for Regresson Leas Squares, Gauss Markov heorem Recursve Leas Squares Lecure 6: RLSC - Prof. Sehu Vjayakumar Lnear Regresson
More information8-node quadrilateral element. Numerical integration
Fnt Elmnt Mthod lctur nots _nod quadrlatral lmnt Pag of 0 -nod quadrlatral lmnt. Numrcal ntgraton h tchnqu usd for th formulaton of th lnar trangl can b formall tndd to construct quadrlatral lmnts as wll
More informationSafety and Reliability of Embedded Systems. (Sicherheit und Zuverlässigkeit eingebetteter Systeme) Stochastic Reliability Analysis
(Schrh und Zuvrlässgk ngbr Sysm) Sochasc Rlably Analyss Conn Dfnon of Rlably Hardwar- vs. Sofwar Rlably Tool Asssd Rlably Modlng Dscrpons of Falurs ovr Tm Rlably Modlng Exampls of Dsrbuon Funcons Th xponnal
More informationTheoretical Seismology
Thorcal Ssmology Lcur 9 Sgnal Procssng Fourr analyss Fourr sudd a h Écol Normal n Pars, augh by Lagrang, who Fourr dscrbd as h frs among Europan mn of scnc, Laplac, who Fourr rad lss hghly, and by Mong.
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 4
CS434a/54a: Paern Recognon Prof. Olga Veksler Lecure 4 Oulne Normal Random Varable Properes Dscrmnan funcons Why Normal Random Varables? Analycally racable Works well when observaon comes form a corruped
More informationEXERCISE - 01 CHECK YOUR GRASP
DIFFERENTIAL EQUATION EXERCISE - CHECK YOUR GRASP 7. m hn D() m m, D () m m. hn givn D () m m D D D + m m m m m m + m m m m + ( m ) (m ) (m ) (m + ) m,, Hnc numbr of valus of mn will b. n ( ) + c sinc
More informationPupil / Class Record We can assume a word has been learned when it has been either tested or used correctly at least three times.
2 Pupi / Css Rr W ssum wr hs b r wh i hs b ihr s r us rry s hr ims. Nm: D Bu: fr i bus brhr u firs hf hp hm s uh i iv iv my my mr muh m w ih w Tik r pp push pu sh shu sisr s sm h h hir hr hs im k w vry
More informationMicroscopic Flow Characteristics Time Headway - Distribution
CE57: Traffic Flow Thory Spring 20 Wk 2 Modling Hadway Disribuion Microscopic Flow Characrisics Tim Hadway - Disribuion Tim Hadway Dfiniion Tim Hadway vrsus Gap Ahmd Abdl-Rahim Civil Enginring Dparmn,
More informationSingle-loop System Reliability-Based Design & Topology Optimization (SRBDO/SRBTO): A Matrix-based System Reliability (MSR) Method
10 h US Naonal Congress on Compuaonal Mechancs Columbus, Oho 16-19, 2009 Sngle-loop Sysem Relably-Based Desgn & Topology Opmzaon (SRBDO/SRBTO): A Marx-based Sysem Relably (MSR) Mehod Tam Nguyen, Junho
More informationS.Y. B.Sc. (IT) : Sem. III. Applied Mathematics. Q.1 Attempt the following (any THREE) [15]
S.Y. B.Sc. (IT) : Sm. III Applid Mahmaics Tim : ½ Hrs.] Prlim Qusion Papr Soluion [Marks : 75 Q. Amp h following (an THREE) 3 6 Q.(a) Rduc h mari o normal form and find is rank whr A 3 3 5 3 3 3 6 Ans.:
More informationProblem 1: Consider the following stationary data generation process for a random variable y t. e t ~ N(0,1) i.i.d.
A/CN C m Sr Anal Profor Òcar Jordà Wnr conomc.c. Dav POBLM S SOLIONS Par I Analcal Quon Problm : Condr h followng aonar daa gnraon proc for a random varabl - N..d. wh < and N -. a Oban h populaon man varanc
More informationNetwork Congestion Games
Ntwork Congstion Gams Assistant Profssor Tas A&M Univrsity Collg Station, TX TX Dallas Collg Station Austin Houston Bst rout dpnds on othrs Ntwork Congstion Gams Travl tim incrass with congstion Highway
More informationNAME: ANSWER KEY DATE: PERIOD. DIRECTIONS: MULTIPLE CHOICE. Choose the letter of the correct answer.
R A T T L E R S S L U G S NAME: ANSWER KEY DATE: PERIOD PREAP PHYSICS REIEW TWO KINEMATICS / GRAPHING FORM A DIRECTIONS: MULTIPLE CHOICE. Chs h r f h rr answr. Us h fgur bw answr qusns 1 and 2. 0 10 20
More informationSafety and Reliability of Embedded Systems. (Sicherheit und Zuverlässigkeit eingebetteter Systeme) Stochastic Reliability Analysis
Safy and Rlably of Embddd Sysms (Schrh und Zuvrlässgk ngbr Sysm) Sochasc Rlably Analyss Safy and Rlably of Embddd Sysms Conn Dfnon of Rlably Hardwar- vs. Sofwar Rlably Tool Asssd Rlably Modlng Dscrpons
More informationGradient Descent for General Reinforcement Learning
To appar n M. S. Karns, S. A. Solla, and D. A. Cohn, dors, Advancs n Nral Informaon Procssng Sysms, MIT Prss, Cambrdg, MA, 999. Gradn Dscn for Gnral Rnforcmn Larnng Lmon Bard Andrw Moor lmon@cs.cm.d awm@cs.cm.d
More informationVariants of Pegasos. December 11, 2009
Inroducon Varans of Pegasos SooWoong Ryu bshboy@sanford.edu December, 009 Youngsoo Cho yc344@sanford.edu Developng a new SVM algorhm s ongong research opc. Among many exng SVM algorhms, we wll focus on
More informationBethe-Salpeter Equation Green s Function and the Bethe-Salpeter Equation for Effective Interaction in the Ladder Approximation
Bh-Salp Equaon n s Funcon and h Bh-Salp Equaon fo Effcv Inacon n h Ladd Appoxmaon Csa A. Z. Vasconcllos Insuo d Físca-UFRS - upo: Físca d Hadons Sngl-Pacl Popagao. Dagam xpanson of popagao. W consd as
More information4.1 The Uniform Distribution Def n: A c.r.v. X has a continuous uniform distribution on [a, b] when its pdf is = 1 a x b
4. Th Uniform Disribuion Df n: A c.r.v. has a coninuous uniform disribuion on [a, b] whn is pdf is f x a x b b a Also, b + a b a µ E and V Ex4. Suppos, h lvl of unblivabiliy a any poin in a Transformrs
More informationCIVL 8/ D Boundary Value Problems - Triangular Elements (T6) 1/8
CIVL 8/7 -D Boundar Valu Problm - rangular Elmn () /8 SI-ODE RIAGULAR ELEMES () A quadracall nrpolad rangular lmn dfnd b nod, hr a h vrc and hr a h mddl a ach d. h mddl nod, dpndng on locaon, ma dfn a
More informationControl System Engineering (EE301T) Assignment: 2
Conrol Sysm Enginring (EE0T) Assignmn: PART-A (Tim Domain Analysis: Transin Rspons Analysis). Oain h rspons of a uniy fdack sysm whos opn-loop ransfr funcion is (s) s ( s 4) for a uni sp inpu and also
More information(heat loss divided by total enthalpy flux) is of the order of 8-16 times
16.51, Rok Prolson Prof. Manl Marnz-Sanhz r 8: Convv Ha ransfr: Ohr Effs Ovrall Ha oss and Prforman Effs of Ha oss (1) Ovrall Ha oss h loal ha loss r n ara s q = ρ ( ) ngrad ha loss s a S, and sng m =
More informationEEC 686/785 Modeling & Performance Evaluation of Computer Systems. Lecture 12
EEC 686/785 Modlng & Prformanc Evaluaton of Computr Systms Lctur Dpartmnt of Elctrcal and Computr Engnrng Clvland Stat Unvrsty wnbng@.org (basd on Dr. Ra Jan s lctur nots) Outln Rvw of lctur k r Factoral
More informationLecture 1: Numerical Integration The Trapezoidal and Simpson s Rule
Lcur : Numrical ngraion Th Trapzoidal and Simpson s Rul A problm Th probabiliy of a normally disribud (man µ and sandard dviaion σ ) vn occurring bwn h valus a and b is B A P( a x b) d () π whr a µ b -
More informationMathematical Statistics. Chapter VIII Sampling Distributions and the Central Limit Theorem
Mahmacal ascs 8 Chapr VIII amplg Dsrbos ad h Cral Lm Thorm Fcos of radom arabls ar sall of rs sascal applcao Cosdr a s of obsrabl radom arabls L For ampl sppos h arabls ar a radom sampl of s from a poplao
More informationOn the Derivatives of Bessel and Modified Bessel Functions with Respect to the Order and the Argument
Inrnaional Rsarch Journal of Applid Basic Scincs 03 Aailabl onlin a wwwirjabscom ISSN 5-838X / Vol 4 (): 47-433 Scinc Eplorr Publicaions On h Driais of Bssl Modifid Bssl Funcions wih Rspc o h Ordr h Argumn
More informationReliability analysis of time - dependent stress - strength system when the number of cycles follows binomial distribution
raoal Joural of Sascs ad Ssms SSN 97-675 Volum, Numbr 7,. 575-58 sarch da Publcaos h://www.rublcao.com labl aalss of m - dd srss - srgh ssm wh h umbr of ccls follows bomal dsrbuo T.Sumah Umamahswar, N.Swah,
More informationCharging of capacitor through inductor and resistor
cur 4&: R circui harging of capacior hrough inducor and rsisor us considr a capacior of capacianc is conncd o a D sourc of.m.f. E hrough a rsisr of rsisanc R, an inducor of inducanc and a y K in sris.
More informationDiscrete Markov Process. Introduction. Example: Balls and Urns. Stochastic Automaton. INTRODUCTION TO Machine Learning 3rd Edition
EHEM ALPAYDI he MI Press, 04 Lecure Sldes for IRODUCIO O Machne Learnng 3rd Edon alpaydn@boun.edu.r hp://www.cmpe.boun.edu.r/~ehem/ml3e Sldes from exboo resource page. Slghly eded and wh addonal examples
More informationTransient Analysis of Two-dimensional State M/G/1 Queueing Model with Multiple Vacations and Bernoulli Schedule
Inrnaonal Journal of Compur Applcaons (975 8887) Volum 4 No.3, Fbruary 22 Transn Analyss of Two-dmnsonal Sa M/G/ Quung Modl wh Mulpl Vacaons and Brnoull Schdul Indra Assoca rofssor Dparmn of Sascs and
More information1. Inverse Matrix 4[(3 7) (02)] 1[(0 7) (3 2)] Recall that the inverse of A is equal to:
Rfrncs Brnank, B. and I. Mihov (1998). Masuring monary policy, Quarrly Journal of Economics CXIII, 315-34. Blanchard, O. R. Proi (00). An mpirical characrizaion of h dynamic ffcs of changs in govrnmn spnding
More informationST 524 NCSU - Fall 2008 One way Analysis of variance Variances not homogeneous
ST 54 NCSU - Fall 008 On way Analyss of varanc Varancs not homognous On way Analyss of varanc Exampl (Yandll, 997) A plant scntst masurd th concntraton of a partcular vrus n plant sap usng ELISA (nzym-lnkd
More informationEnsemble Methods: Boosting
Ensemble Methods: Boostng Ncholas Ruozz Unversty of Texas at Dallas Based on the sldes of Vbhav Gogate and Rob Schapre Last Tme Varance reducton va baggng Generate new tranng data sets by samplng wth replacement
More informationMidterm Examination (100 pts)
Econ 509 Spring 2012 S.L. Parn Midrm Examinaion (100 ps) Par I. 30 poins 1. Dfin h Law of Diminishing Rurns (5 ps.) Incrasing on inpu, call i inpu x, holding all ohr inpus fixd, on vnuall runs ino h siuaion
More informationwhere: u: input y: output x: state vector A, B, C, D are const matrices
Sa pac modl: linar: y or in om : Sa q : f, u Oupu q : y h, u u Du F Gu y H Ju whr: u: inpu y: oupu : a vcor,,, D ar con maric Eampl " $ & ' " $ & 'u y " & * * * * [ ],, D H D I " $ " & $ ' " & $ ' " &
More informationThe Fourier Transform
/9/ Th ourr Transform Jan Baptst Josph ourr 768-83 Effcnt Data Rprsntaton Data can b rprsntd n many ways. Advantag usng an approprat rprsntaton. Eampls: osy ponts along a ln Color spac rd/grn/blu v.s.
More informationSolution in semi infinite diffusion couples (error function analysis)
Soluon n sem nfne dffuson couples (error funcon analyss) Le us consder now he sem nfne dffuson couple of wo blocks wh concenraon of and I means ha, n a A- bnary sysem, s bondng beween wo blocks made of
More informationAn introduction to Support Vector Machine
An nroducon o Suppor Vecor Machne 報告者 : 黃立德 References: Smon Haykn, "Neural Neworks: a comprehensve foundaon, second edon, 999, Chaper 2,6 Nello Chrsann, John Shawe-Tayer, An Inroducon o Suppor Vecor Machnes,
More informationMor Tutorial at www.dumblittldoctor.com Work th problms without a calculator, but us a calculator to chck rsults. And try diffrntiating your answrs in part III as a usful chck. I. Applications of Intgration
More informationSelf-interaction mass formula that relates all leptons and quarks to the electron
Slf-intraction mass formula that rlats all lptons and quarks to th lctron GERALD ROSEN (a) Dpartmnt of Physics, Drxl Univrsity Philadlphia, PA 19104, USA PACS. 12.15. Ff Quark and lpton modls spcific thoris
More informationIntroduction to logistic regression
Itroducto to logstc rgrsso Gv: datast D { 2 2... } whr s a k-dmsoal vctor of ral-valud faturs or attrbuts ad s a bar class labl or targt. hus w ca sa that R k ad {0 }. For ampl f k 4 a datast of 3 data
More informationChap 2: Reliability and Availability Models
Chap : lably ad valably Modls lably = prob{s s fully fucog [,]} Suppos from [,] m prod, w masur ou of N compos, of whch N : # of compos oprag corrcly a m N f : # of compos whch hav fald a m rlably of h
More informationIn the complete model, these slopes are ANALYSIS OF VARIANCE FOR THE COMPLETE TWO-WAY MODEL. (! i+1 -! i ) + [(!") i+1,q - [(!
ANALYSIS OF VARIANCE FOR THE COMPLETE TWO-WAY MODEL The frs hng o es n wo-way ANOVA: Is here neracon? "No neracon" means: The man effecs model would f. Ths n urn means: In he neracon plo (wh A on he horzonal
More informationCHAPTER CHAPTER14. Expectations: The Basic Tools. Prepared by: Fernando Quijano and Yvonn Quijano
Expcaions: Th Basic Prpard by: Frnando Quijano and Yvonn Quijano CHAPTER CHAPTER14 2006 Prnic Hall Businss Publishing Macroconomics, 4/ Olivir Blanchard 14-1 Today s Lcur Chapr 14:Expcaions: Th Basic Th
More informationEE 434 Lecture 22. Bipolar Device Models
EE 434 Lcur 22 Bipolar Dvic Modls Quiz 14 Th collcor currn of a BJT was masurd o b 20mA and h bas currn masurd o b 0.1mA. Wha is h fficincy of injcion of lcrons coming from h mir o h collcor? 1 And h numbr
More informationBasic Polyhedral theory
Basic Polyhdral thory Th st P = { A b} is calld a polyhdron. Lmma 1. Eithr th systm A = b, b 0, 0 has a solution or thr is a vctorπ such that π A 0, πb < 0 Thr cass, if solution in top row dos not ist
More informationEE243 Advanced Electromagnetic Theory Lec # 10: Poynting s Theorem, Time- Harmonic EM Fields
Appl M Fall 6 Nuruhr Lcur # r 9/6/6 4 Avanc lcromagnc Thory Lc # : Poynng s Thorm Tm- armonc M Fls Poynng s Thorm Consrvaon o nrgy an momnum Poynng s Thorm or Lnar sprsv Ma Poynng s Thorm or Tm-armonc
More informationChapter 6 Student Lecture Notes 6-1
Chaptr 6 Studnt Lctur Nots 6-1 Chaptr Goals QM353: Busnss Statstcs Chaptr 6 Goodnss-of-Ft Tsts and Contngncy Analyss Aftr compltng ths chaptr, you should b abl to: Us th ch-squar goodnss-of-ft tst to dtrmn
More informationOn the Speed of Heat Wave. Mihály Makai
On h Spd of Ha Wa Mihály Maai maai@ra.bm.hu Conns Formulaion of h problm: infini spd? Local hrmal qulibrium (LTE hypohsis Balanc quaion Phnomnological balanc Spd of ha wa Applicaion in plasma ranspor 1.
More informationMixture Ratio Estimators Using Multi-Auxiliary Variables and Attributes for Two-Phase Sampling
Opn Journal of Sascs 04 4 776-788 Publshd Onln Ocobr 04 n Scs hp://scrporg/ournal/os hp://ddoorg/0436/os0449073 Mur ao Esmaors Usng Mul-Aular Varabls and Arbus for To-Phas Samplng Paul Mang Waru John Kung
More informationTSS = SST + SSE An orthogonal partition of the total SS
ANOVA: Topc 4. Orhogonal conrass [ST&D p. 183] H 0 : µ 1 = µ =... = µ H 1 : The mean of a leas one reamen group s dfferen To es hs hypohess, a basc ANOVA allocaes he varaon among reamen means (SST) equally
More informationFolding of Regular CW-Complexes
Ald Mathmatcal Scncs, Vol. 6,, no. 83, 437-446 Foldng of Rgular CW-Comlxs E. M. El-Kholy and S N. Daoud,3. Dartmnt of Mathmatcs, Faculty of Scnc Tanta Unvrsty,Tanta,Egyt. Dartmnt of Mathmatcs, Faculty
More informationSER/BER in a Fading Channel
SER/BER in a Fading Channl Major points for a fading channl: * SNR is a R.V. or R.P. * SER(BER) dpnds on th SNR conditional SER(BER). * Two prformanc masurs: outag probability and avrag SER(BER). * Ovrall,
More informationDeift/Zhou Steepest descent, Part I
Lctur 9 Dift/Zhou Stpst dscnt, Part I W now focus on th cas of orthogonal polynomials for th wight w(x) = NV (x), V (x) = t x2 2 + x4 4. Sinc th wight dpnds on th paramtr N N w will writ π n,n, a n,n,
More informationPhysics of Very High Frequency (VHF) Capacitively Coupled Plasma Discharges
Physcs of Vry Hgh Frquncy (VHF) Capactvly Coupld Plasma Dschargs Shahd Rauf, Kallol Bra, Stv Shannon, and Kn Collns Appld Matrals, Inc., Sunnyval, CA AVS 54 th Intrnatonal Symposum Sattl, WA Octobr 15-19,
More informationFirst derivative analysis
Robrto s Nots on Dirntial Calculus Chaptr 8: Graphical analysis Sction First drivativ analysis What you nd to know alrady: How to us drivativs to idntiy th critical valus o a unction and its trm points
More informationMA1506 Tutorial 2 Solutions. Question 1. (1a) 1 ) y x. e x. 1 exp (in general, Integrating factor is. ye dx. So ) (1b) e e. e c.
MA56 utorial Solutions Qustion a Intgrating fator is ln p p in gnral, multipl b p So b ln p p sin his kin is all a Brnoulli quation -- st Sin w fin Y, Y Y, Y Y p Qustion Dfin v / hn our quation is v μ
More informationInstitute of Actuaries of India
Insiu of Acuaris of India ubjc CT3 Probabiliy and Mahmaical aisics Novmbr Examinaions INDICATIVE OLUTION Pag of IAI CT3 Novmbr ol. a sampl man = 35 sampl sandard dviaion = 36.6 b for = uppr bound = 35+*36.6
More informationElementary Differential Equations and Boundary Value Problems
Elmnar Diffrnial Equaions and Boundar Valu Problms Boc. & DiPrima 9 h Ediion Chapr : Firs Ordr Diffrnial Equaions 00600 คณ ตศาสตร ว ศวกรรม สาขาว ชาว ศวกรรมคอมพ วเตอร ป การศ กษา /55 ผศ.ดร.อร ญญา ผศ.ดร.สมศ
More informationDavisson Germer experiment Announcements:
Davisson Grmr xprimnt Announcmnts: Homwork st 7 is du Wdnsday. Problm solving sssions M3-5, T3-5. Th 2 nd midtrm will b April 7 in MUEN E0046 at 7:30pm. BFFs: Davisson and Grmr. Today w will go ovr th
More informationShortest Paths in Graphs. Paths in graphs. Shortest paths CS 445. Alon Efrat Slides courtesy of Erik Demaine and Carola Wenk
S 445 Shortst Paths n Graphs lon frat Sls courtsy of rk man an arola Wnk Paths n raphs onsr a raph G = (V, ) wth -wht functon w : R. Th wht of path p = v v v k s fn to xampl: k = w ( p) = w( v, v + ).
More informationCmd> data<-matread("hwprobs.dat","exmpl8.2") This is the data shown in Table 8.5, the copper and diet factors.
Sa 5303 (Ohlr): Facoral Hanou 3 Cm> aa
More informationSupplementary Materials
6 Supplmntary Matrials APPENDIX A PHYSICAL INTERPRETATION OF FUEL-RATE-SPEED FUNCTION A truck running on a road with grad/slop θ positiv if moving up and ngativ if moving down facs thr rsistancs: arodynamic
More informationOutline. Probabilistic Model Learning. Probabilistic Model Learning. Probabilistic Model for Time-series Data: Hidden Markov Model
Probablsc Model for Tme-seres Daa: Hdden Markov Model Hrosh Mamsuka Bonformacs Cener Kyoo Unversy Oulne Three Problems for probablsc models n machne learnng. Compung lkelhood 2. Learnng 3. Parsng (predcon
More informationAnalysis of decentralized potential field based multi-agent navigation via primal-dual Lyapunov theory
Analyss of dcnralzd ponal fld basd mul-agn navgaon va prmal-dual Lyapunov hory Th MIT Faculy has mad hs arcl opnly avalabl. Plas shar how hs accss bnfs you. Your sory mars. Caon As Publshd Publshr Dmarogonas,
More information