Variants of Pegasos. December 11, 2009

Size: px
Start display at page:

Download "Variants of Pegasos. December 11, 2009"

Transcription

1 Inroducon Varans of Pegasos SooWoong Ryu December, 009 Youngsoo Cho Developng a new SVM algorhm s ongong research opc. Among many exng SVM algorhms, we wll focus on Pegasos. Pegasos uses alernang eraon schemes, whch are by alernang sochasc graden descen seps and projecon sep. The Pegasos can solve a ex classfcaon problem from Reuers Corpus Volume wh 800,000 ranng examples n 5 seconds. We wll examne he algorhms robusness by applyng he algorhm o raher smple ex classfcaon problem (namely, he one gven n he cs9 homework. We also presen hree possble varans of Pegasos; namely, undo mehod, wo-ou mehod and probably mehod. We wll examne con and pro of each mehod. Algorhm descrpon. Pegasos The Pegasos s a subgraden based algorhm ha mnmzes he followng unconsraned srcly convex objecve funcon. λ f( ω = ω + l( ω;( x, y ( m ( x, y S where l( ω;( x, y = max{0, y ω, x } ( However, does no use eq( as an objecve funcon a each eraon. Insead, uses λ f( ω; A = ω + l( ω;( x, y (3 k ( x, y A where a se A S whose sze s k and S, a ranng se. The Pegasos consss of wo major seps n updang ω :. subseepes graden updae ω + and. projecon sep. Specfcally, n each eraon, he Pegasos looks for a seepes graden search drecon gven A and ses he learnng rae o beη = λ. Then, projecs he updaed ω + no he ball, B = { ω : ω }. The reason for projecon s ha we already know he opmal sqr ( λ soluon,ω, resdes n B.. A Varan of Pegasos A key pon n a varan of pegasos s how o deal wh oulers n ranng se, S. There are oulers n S. These oulers resul n ncrease n objecve funcon value n some eraon snce pegasos chooses A..d. from S. To preven hs, hree mehods are used;. Undo mehod. wo-ou mehod and 3. probably mehod. A modfcaon from pegasos s appled a he end of eraon when ω + s compued. In he undo mehod, wh compuedω +, he new objecve funcon value, f ( ω +, s also compued. If f ( ω+ > f ( ω hen we do no use compuedω +. Insead we le ω + o be equal oω. If f ( ω+ f ( ω, hen we keep

2 ω + as updaed. In he wo-ou mehod, he ranng se, S, s updaed a each eraon. If f ( ω+ > f ( ω, hen we do no use compue ω + and also append a flag o each daa, ( x, y n A + and coun he number of flags n each daa n A +. If he number of flags n a daa becomes wo (.e n f ( x, y =, he daa s aken away from S +. Le D + o be a se of hose daa ha are aken away from S +. Then, we oban S+ = S+ D +. A he eraon of +, S + s used as a ranng se. Agan, f f ( ω+ f ( ω, hen we keep ω + as updaed as n undo mehod. The probably mehod s smlar o he wo-ou mehod. The dfference s n how o updae S. In he probably mehod, for each daa, ( x, y, he probably of beng sampled o A a eraon (le s call ha probably p( x, y s se o n he begnnng. As he eraon proceeds, hese probables are updaed by examnng he objecve funcon value. If f ( ω+ > f ( ω, hen we decrease p( x, y for each ( x, y n A. The varan algorhm s shown below. INPUT: S, λ, T, k INITIALIZE: Choose w s.. FOR =,,3,..., T Choose A w λ S, where A = k. Se A + = {( x, y A : y w, x < } η =, w = ( η λ w + + yx η Se + k ( x, y A IF f ω+ > Se ω = + ELSE ( f ( ω ω FOR =,,3,..., Sze( A n ( x, y = n ( x, y + f f IF n f ( x, y = Se D = D {( x, y } + + ω = mn{, } w / λ Se + + w + λ S = S D, OUTPUT: ω + 3 Resul Frs of all, we downloaded he Pegasos source code from hp:// shas/code/ndex.hml. There are hree sample ses of ranng daa and one es daa se: One ranng se of sze 000, wo oher ses of musch smaller sze and a es se of sze 600. One neresng characersc of hs daa se s ha l norm of every daa s one. Ths pus all he daa ono he surface of spheres. We needed o look for more general daa ses whch do no exhb such a characersc. Thus, we use he ranng se and es ses gven for cs9 homework #. The sze of ranng se s 44 and es se has sze of 800. One of he neresng characerscs of he Pegasos s ha we have a conrol on he sze of A, whch s k. If k =, hen Pegasos s very smlar o sochasc graden mehod. If k = S,

3 hen resuls n a modfed graden-descen algorhm. 3. undo mehod Wha he undo mehod acually dose n subgraden based algorhm s o preven he ascen drecon. Please see fg.( and (. In he begnnng of he eraons, he undo mehod seems o do much beer han he pegasos. However, as he eraon reaches he pon close o he convergence regon, he pegasos does beer han undo mehod. Ths resul s analogs o he comparson beween seepes descen algorhm and conjugae graden mehod. Alhough he seepes descen may exhb beer performance n he begnnng of eraons, he one ha wns he performance s conjugae graden mehod. The lesson here s ha allowng ascen drecon somemes can be poson a ha momen, bu can be medcne. As descrbed n algorhm descrpon secon, we combned he undo, wo-ou, and probably mehod and he resul s ha he undo mehod s domnan n choosng he search drecon. In fg.(3 and (4, all he hree mehods (.e. undo mehod, undo-probably combned mehod, and undo-probably-wo ou combned mehod exhb almos dencal performance. 3. wo-ou mehod Snce we ve found ha he undo mehod should no be used, we decded no o use n boh wo-ou mehod and probably mehod. Thus, n wo-ou mehod, alhough f ( ω+ > f ( ω n some, we keep he w+ as updaed. Only hng we change s o updae S every eraon by elmnang daa whose flag s equal o wo. The resul s almos dencal o he orgnal pegasos mehod. (.e. see he fg.(5 and (6. We srongly suspec ha he reason for he almos dencal resul s ha he ranng se we have (.e. he sze of abou 000 s oo small o ake an advanage of wo-ou mehod. To ake an advanage of wo-ou mehod, we need enough eraon o ge rd of oulers. The average number of daa aken away from he ranng se unl convergence s around 80. Among 80 daa, here may be good daa n. I s because akng away he daa whose number of flag s wo s oo cruel. Thus, we ook away he daa whose number of flag s four. Then, however, he number of daa aken away from he ranng se s very small (.e. zero, one or wo unl he convergence. The reason ha we have hope n hs mehod s ha even wh our ranng se, he wo-ou mehod exhbs slghly beer performance han he pegasos rgh before he convergence occur (.e. see fg.(7. However, hs s smply our hope and need o be proven wh bgger ranng se n he fuure. 3.3 probably mehod As n case of he wo-ou mehod, he probably mehod exhbs almos dencal resul o pegasos (.e. see fg.(8 and (9. Ths also leaves us a hope ha he probably mehod may 3 works beer han pegasos when he ranng se sze s much bgger. Fgure. Comparson beween pegasos and undo mehod Fgure. Comparson beween pegasos and undo mehod

4 Fgure3. undo mehod s domnan n choosng search drecon Fgure4. undo mehod s domnan n choosng search drecon Fgure5. comparson beween pegasos and wo-ou mehod Fgure6. comparson beween pegasos and wo-ou mehod Fgure7. zoomed n comparson beween pegasos and wo-ou mehod

5 Fgure8. comparson beween pegasos and probably mehod Fgure9. comparson beween pegasos and probably mehod 4 Concluson Developng a new algorhm s a very hard job. Afer readng many papers and geng very deep nsgh abou he algorhm, hen fnally someone can develop wha s so called noble algorhm. Throughou he projec, we have red many oher algorhms whch have no been presened n hs repor. However he resuls were all pessmsc. Inally, for example, we red o exrac only suppor vecors from ranng se o ncrease he convergence rae. However, due o he presence of oulers, when we exrac he suppor vecors (SVs, oulers were also exraced wh SVs. Ths caused even worse es error han pegasos gave. In hs repor, we presened hree dfferen mehods as a varan of Pegasos: undo mehod, wo-ou mehod and probably mehod. The undo mehod aced lke seepes descen mehod. I may converge o a good soluon, bu n oher mes, would no converge o a good soluon. Boh wo-ou mehod and probably mehod gve a way of fndng oulers n ranng se. Takng away daa ha ges wo flags durng he smulaons, we may have hgh rsk of gvng away a good daa. Ths resul can be fxed eher by ncreasng wo o four or fve. However o accomplsh hs, we need a bgger ranng se. The probably mehod also shows he poenal o be used n bgger ranng se. 5 Reference. Shalev-Shwarz, S., Snger, Y. and Srebro, N. (007. Pegasos: Prmal Esmaed sub-graden Solver for SVM. Proceedngs of he 4 h Inernaonal Conference on Machne Learnng.. Joachms, T. (999. Makng Large-Scale SVM Learnng Praccal. MIT Press Cambrdge, MA, USA 3. Joachms, T. (006. Tranng Lnear SVMs n Lnear Tme. Proceedngs of he h ACM SIGKDD nernaonal conference on Knowledge dscovery and daa mnng

CHAPTER 10: LINEAR DISCRIMINATION

CHAPTER 10: LINEAR DISCRIMINATION CHAPER : LINEAR DISCRIMINAION Dscrmnan-based Classfcaon 3 In classfcaon h K classes (C,C,, C k ) We defned dscrmnan funcon g j (), j=,,,k hen gven an es eample, e chose (predced) s class label as C f g

More information

Advanced Machine Learning & Perception

Advanced Machine Learning & Perception Advanced Machne Learnng & Percepon Insrucor: Tony Jebara SVM Feaure & Kernel Selecon SVM Eensons Feaure Selecon (Flerng and Wrappng) SVM Feaure Selecon SVM Kernel Selecon SVM Eensons Classfcaon Feaure/Kernel

More information

Clustering (Bishop ch 9)

Clustering (Bishop ch 9) Cluserng (Bshop ch 9) Reference: Daa Mnng by Margare Dunham (a slde source) 1 Cluserng Cluserng s unsupervsed learnng, here are no class labels Wan o fnd groups of smlar nsances Ofen use a dsance measure

More information

Cubic Bezier Homotopy Function for Solving Exponential Equations

Cubic Bezier Homotopy Function for Solving Exponential Equations Penerb Journal of Advanced Research n Compung and Applcaons ISSN (onlne: 46-97 Vol. 4, No.. Pages -8, 6 omoopy Funcon for Solvng Eponenal Equaons S. S. Raml *,,. Mohamad Nor,a, N. S. Saharzan,b and M.

More information

Lecture 6: Learning for Control (Generalised Linear Regression)

Lecture 6: Learning for Control (Generalised Linear Regression) Lecure 6: Learnng for Conrol (Generalsed Lnear Regresson) Conens: Lnear Mehods for Regresson Leas Squares, Gauss Markov heorem Recursve Leas Squares Lecure 6: RLSC - Prof. Sehu Vjayakumar Lnear Regresson

More information

Lecture VI Regression

Lecture VI Regression Lecure VI Regresson (Lnear Mehods for Regresson) Conens: Lnear Mehods for Regresson Leas Squares, Gauss Markov heorem Recursve Leas Squares Lecure VI: MLSC - Dr. Sehu Vjayakumar Lnear Regresson Model M

More information

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 4

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 4 CS434a/54a: Paern Recognon Prof. Olga Veksler Lecure 4 Oulne Normal Random Varable Properes Dscrmnan funcons Why Normal Random Varables? Analycally racable Works well when observaon comes form a corruped

More information

[ ] 2. [ ]3 + (Δx i + Δx i 1 ) / 2. Δx i-1 Δx i Δx i+1. TPG4160 Reservoir Simulation 2018 Lecture note 3. page 1 of 5

[ ] 2. [ ]3 + (Δx i + Δx i 1 ) / 2. Δx i-1 Δx i Δx i+1. TPG4160 Reservoir Simulation 2018 Lecture note 3. page 1 of 5 TPG460 Reservor Smulaon 08 page of 5 DISCRETIZATIO OF THE FOW EQUATIOS As we already have seen, fne dfference appromaons of he paral dervaves appearng n he flow equaons may be obaned from Taylor seres

More information

Introduction to Boosting

Introduction to Boosting Inroducon o Boosng Cynha Rudn PACM, Prnceon Unversy Advsors Ingrd Daubeches and Rober Schapre Say you have a daabase of news arcles, +, +, -, -, +, +, -, -, +, +, -, -, +, +, -, + where arcles are labeled

More information

Solution in semi infinite diffusion couples (error function analysis)

Solution in semi infinite diffusion couples (error function analysis) Soluon n sem nfne dffuson couples (error funcon analyss) Le us consder now he sem nfne dffuson couple of wo blocks wh concenraon of and I means ha, n a A- bnary sysem, s bondng beween wo blocks made of

More information

Robust and Accurate Cancer Classification with Gene Expression Profiling

Robust and Accurate Cancer Classification with Gene Expression Profiling Robus and Accurae Cancer Classfcaon wh Gene Expresson Proflng (Compuaonal ysems Bology, 2005) Auhor: Hafeng L, Keshu Zhang, ao Jang Oulne Background LDA (lnear dscrmnan analyss) and small sample sze problem

More information

Outline. Probabilistic Model Learning. Probabilistic Model Learning. Probabilistic Model for Time-series Data: Hidden Markov Model

Outline. Probabilistic Model Learning. Probabilistic Model Learning. Probabilistic Model for Time-series Data: Hidden Markov Model Probablsc Model for Tme-seres Daa: Hdden Markov Model Hrosh Mamsuka Bonformacs Cener Kyoo Unversy Oulne Three Problems for probablsc models n machne learnng. Compung lkelhood 2. Learnng 3. Parsng (predcon

More information

In the complete model, these slopes are ANALYSIS OF VARIANCE FOR THE COMPLETE TWO-WAY MODEL. (! i+1 -! i ) + [(!") i+1,q - [(!

In the complete model, these slopes are ANALYSIS OF VARIANCE FOR THE COMPLETE TWO-WAY MODEL. (! i+1 -! i ) + [(!) i+1,q - [(! ANALYSIS OF VARIANCE FOR THE COMPLETE TWO-WAY MODEL The frs hng o es n wo-way ANOVA: Is here neracon? "No neracon" means: The man effecs model would f. Ths n urn means: In he neracon plo (wh A on he horzonal

More information

Advanced time-series analysis (University of Lund, Economic History Department)

Advanced time-series analysis (University of Lund, Economic History Department) Advanced me-seres analss (Unvers of Lund, Economc Hsor Dearmen) 3 Jan-3 Februar and 6-3 March Lecure 4 Economerc echnues for saonar seres : Unvarae sochasc models wh Box- Jenns mehodolog, smle forecasng

More information

Appendix H: Rarefaction and extrapolation of Hill numbers for incidence data

Appendix H: Rarefaction and extrapolation of Hill numbers for incidence data Anne Chao Ncholas J Goell C seh lzabeh L ander K Ma Rober K Colwell and Aaron M llson 03 Rarefacon and erapolaon wh ll numbers: a framewor for samplng and esmaon n speces dversy sudes cology Monographs

More information

V.Abramov - FURTHER ANALYSIS OF CONFIDENCE INTERVALS FOR LARGE CLIENT/SERVER COMPUTER NETWORKS

V.Abramov - FURTHER ANALYSIS OF CONFIDENCE INTERVALS FOR LARGE CLIENT/SERVER COMPUTER NETWORKS R&RATA # Vol.) 8, March FURTHER AALYSIS OF COFIDECE ITERVALS FOR LARGE CLIET/SERVER COMPUTER ETWORKS Vyacheslav Abramov School of Mahemacal Scences, Monash Unversy, Buldng 8, Level 4, Clayon Campus, Wellngon

More information

Chapter Lagrangian Interpolation

Chapter Lagrangian Interpolation Chaper 5.4 agrangan Inerpolaon Afer readng hs chaper you should be able o:. dere agrangan mehod of nerpolaon. sole problems usng agrangan mehod of nerpolaon and. use agrangan nerpolans o fnd deraes and

More information

HEAT CONDUCTION PROBLEM IN A TWO-LAYERED HOLLOW CYLINDER BY USING THE GREEN S FUNCTION METHOD

HEAT CONDUCTION PROBLEM IN A TWO-LAYERED HOLLOW CYLINDER BY USING THE GREEN S FUNCTION METHOD Journal of Appled Mahemacs and Compuaonal Mechancs 3, (), 45-5 HEAT CONDUCTION PROBLEM IN A TWO-LAYERED HOLLOW CYLINDER BY USING THE GREEN S FUNCTION METHOD Sansław Kukla, Urszula Sedlecka Insue of Mahemacs,

More information

J i-1 i. J i i+1. Numerical integration of the diffusion equation (I) Finite difference method. Spatial Discretization. Internal nodes.

J i-1 i. J i i+1. Numerical integration of the diffusion equation (I) Finite difference method. Spatial Discretization. Internal nodes. umercal negraon of he dffuson equaon (I) Fne dfference mehod. Spaal screaon. Inernal nodes. R L V For hermal conducon le s dscree he spaal doman no small fne spans, =,,: Balance of parcles for an nernal

More information

DEEP UNFOLDING FOR MULTICHANNEL SOURCE SEPARATION SUPPLEMENTARY MATERIAL

DEEP UNFOLDING FOR MULTICHANNEL SOURCE SEPARATION SUPPLEMENTARY MATERIAL DEEP UNFOLDING FOR MULTICHANNEL SOURCE SEPARATION SUPPLEMENTARY MATERIAL Sco Wsdom, John Hershey 2, Jonahan Le Roux 2, and Shnj Waanabe 2 Deparmen o Elecrcal Engneerng, Unversy o Washngon, Seale, WA, USA

More information

Robustness Experiments with Two Variance Components

Robustness Experiments with Two Variance Components Naonal Insue of Sandards and Technology (NIST) Informaon Technology Laboraory (ITL) Sascal Engneerng Dvson (SED) Robusness Expermens wh Two Varance Componens by Ana Ivelsse Avlés avles@ns.gov Conference

More information

Lecture 11 SVM cont

Lecture 11 SVM cont Lecure SVM con. 0 008 Wha we have done so far We have esalshed ha we wan o fnd a lnear decson oundary whose margn s he larges We know how o measure he margn of a lnear decson oundary Tha s: he mnmum geomerc

More information

Volatility Interpolation

Volatility Interpolation Volaly Inerpolaon Prelmnary Verson March 00 Jesper Andreasen and Bran Huge Danse Mares, Copenhagen wan.daddy@danseban.com brno@danseban.com Elecronc copy avalable a: hp://ssrn.com/absrac=69497 Inro Local

More information

Li An-Ping. Beijing , P.R.China

Li An-Ping. Beijing , P.R.China A New Type of Cpher: DICING_csb L An-Png Bejng 100085, P.R.Chna apl0001@sna.com Absrac: In hs paper, we wll propose a new ype of cpher named DICING_csb, whch s derved from our prevous sream cpher DICING.

More information

Lecture 2 L n i e n a e r a M od o e d l e s

Lecture 2 L n i e n a e r a M od o e d l e s Lecure Lnear Models Las lecure You have learned abou ha s machne learnng Supervsed learnng Unsupervsed learnng Renforcemen learnng You have seen an eample learnng problem and he general process ha one

More information

Ordinary Differential Equations in Neuroscience with Matlab examples. Aim 1- Gain understanding of how to set up and solve ODE s

Ordinary Differential Equations in Neuroscience with Matlab examples. Aim 1- Gain understanding of how to set up and solve ODE s Ordnary Dfferenal Equaons n Neuroscence wh Malab eamples. Am - Gan undersandng of how o se up and solve ODE s Am Undersand how o se up an solve a smple eample of he Hebb rule n D Our goal a end of class

More information

( ) () we define the interaction representation by the unitary transformation () = ()

( ) () we define the interaction representation by the unitary transformation () = () Hgher Order Perurbaon Theory Mchael Fowler 3/7/6 The neracon Represenaon Recall ha n he frs par of hs course sequence, we dscussed he chrödnger and Hesenberg represenaons of quanum mechancs here n he chrödnger

More information

Graduate Macroeconomics 2 Problem set 5. - Solutions

Graduate Macroeconomics 2 Problem set 5. - Solutions Graduae Macroeconomcs 2 Problem se. - Soluons Queson 1 To answer hs queson we need he frms frs order condons and he equaon ha deermnes he number of frms n equlbrum. The frms frs order condons are: F K

More information

10. A.C CIRCUITS. Theoretically current grows to maximum value after infinite time. But practically it grows to maximum after 5τ. Decay of current :

10. A.C CIRCUITS. Theoretically current grows to maximum value after infinite time. But practically it grows to maximum after 5τ. Decay of current : . A. IUITS Synopss : GOWTH OF UNT IN IUIT : d. When swch S s closed a =; = d. A me, curren = e 3. The consan / has dmensons of me and s called he nducve me consan ( τ ) of he crcu. 4. = τ; =.63, n one

More information

Econ107 Applied Econometrics Topic 5: Specification: Choosing Independent Variables (Studenmund, Chapter 6)

Econ107 Applied Econometrics Topic 5: Specification: Choosing Independent Variables (Studenmund, Chapter 6) Econ7 Appled Economercs Topc 5: Specfcaon: Choosng Independen Varables (Sudenmund, Chaper 6 Specfcaon errors ha we wll deal wh: wrong ndependen varable; wrong funconal form. Ths lecure deals wh wrong ndependen

More information

Mechanics Physics 151

Mechanics Physics 151 Mechancs Physcs 5 Lecure 0 Canoncal Transformaons (Chaper 9) Wha We Dd Las Tme Hamlon s Prncple n he Hamlonan formalsm Dervaon was smple δi δ Addonal end-pon consrans pq H( q, p, ) d 0 δ q ( ) δq ( ) δ

More information

Dynamic Team Decision Theory. EECS 558 Project Shrutivandana Sharma and David Shuman December 10, 2005

Dynamic Team Decision Theory. EECS 558 Project Shrutivandana Sharma and David Shuman December 10, 2005 Dynamc Team Decson Theory EECS 558 Proec Shruvandana Sharma and Davd Shuman December 0, 005 Oulne Inroducon o Team Decson Theory Decomposon of he Dynamc Team Decson Problem Equvalence of Sac and Dynamc

More information

Dual Approximate Dynamic Programming for Large Scale Hydro Valleys

Dual Approximate Dynamic Programming for Large Scale Hydro Valleys Dual Approxmae Dynamc Programmng for Large Scale Hydro Valleys Perre Carpener and Jean-Phlppe Chanceler 1 ENSTA ParsTech and ENPC ParsTech CMM Workshop, January 2016 1 Jon work wh J.-C. Alas, suppored

More information

Machine Learning Linear Regression

Machine Learning Linear Regression Machne Learnng Lnear Regresson Lesson 3 Lnear Regresson Bascs of Regresson Leas Squares esmaon Polynomal Regresson Bass funcons Regresson model Regularzed Regresson Sascal Regresson Mamum Lkelhood (ML)

More information

THE PREDICTION OF COMPETITIVE ENVIRONMENT IN BUSINESS

THE PREDICTION OF COMPETITIVE ENVIRONMENT IN BUSINESS THE PREICTION OF COMPETITIVE ENVIRONMENT IN BUSINESS INTROUCTION The wo dmensonal paral dfferenal equaons of second order can be used for he smulaon of compeve envronmen n busness The arcle presens he

More information

Online Supplement for Dynamic Multi-Technology. Production-Inventory Problem with Emissions Trading

Online Supplement for Dynamic Multi-Technology. Production-Inventory Problem with Emissions Trading Onlne Supplemen for Dynamc Mul-Technology Producon-Invenory Problem wh Emssons Tradng by We Zhang Zhongsheng Hua Yu Xa and Baofeng Huo Proof of Lemma For any ( qr ) Θ s easy o verfy ha he lnear programmng

More information

II. Light is a Ray (Geometrical Optics)

II. Light is a Ray (Geometrical Optics) II Lgh s a Ray (Geomercal Opcs) IIB Reflecon and Refracon Hero s Prncple of Leas Dsance Law of Reflecon Hero of Aleandra, who lved n he 2 nd cenury BC, posulaed he followng prncple: Prncple of Leas Dsance:

More information

Reactive Methods to Solve the Berth AllocationProblem with Stochastic Arrival and Handling Times

Reactive Methods to Solve the Berth AllocationProblem with Stochastic Arrival and Handling Times Reacve Mehods o Solve he Berh AllocaonProblem wh Sochasc Arrval and Handlng Tmes Nsh Umang* Mchel Berlare* * TRANSP-OR, Ecole Polyechnque Fédérale de Lausanne Frs Workshop on Large Scale Opmzaon November

More information

Math 128b Project. Jude Yuen

Math 128b Project. Jude Yuen Mah 8b Proec Jude Yuen . Inroducon Le { Z } be a sequence of observed ndependen vecor varables. If he elemens of Z have a on normal dsrbuon hen { Z } has a mean vecor Z and a varancecovarance marx z. Geomercally

More information

Mechanics Physics 151

Mechanics Physics 151 Mechancs Physcs 5 Lecure 9 Hamlonan Equaons of Moon (Chaper 8) Wha We Dd Las Tme Consruced Hamlonan formalsm H ( q, p, ) = q p L( q, q, ) H p = q H q = p H = L Equvalen o Lagrangan formalsm Smpler, bu

More information

Learning Objectives. Self Organization Map. Hamming Distance(1/5) Introduction. Hamming Distance(3/5) Hamming Distance(2/5) 15/04/2015

Learning Objectives. Self Organization Map. Hamming Distance(1/5) Introduction. Hamming Distance(3/5) Hamming Distance(2/5) 15/04/2015 /4/ Learnng Objecves Self Organzaon Map Learnng whou Exaples. Inroducon. MAXNET 3. Cluserng 4. Feaure Map. Self-organzng Feaure Map 6. Concluson 38 Inroducon. Learnng whou exaples. Daa are npu o he syse

More information

Let s treat the problem of the response of a system to an applied external force. Again,

Let s treat the problem of the response of a system to an applied external force. Again, Page 33 QUANTUM LNEAR RESPONSE FUNCTON Le s rea he problem of he response of a sysem o an appled exernal force. Agan, H() H f () A H + V () Exernal agen acng on nernal varable Hamlonan for equlbrum sysem

More information

( ) [ ] MAP Decision Rule

( ) [ ] MAP Decision Rule Announcemens Bayes Decson Theory wh Normal Dsrbuons HW0 due oday HW o be assgned soon Proec descrpon posed Bomercs CSE 90 Lecure 4 CSE90, Sprng 04 CSE90, Sprng 04 Key Probables 4 ω class label X feaure

More information

Mechanics Physics 151

Mechanics Physics 151 Mechancs Physcs 5 Lecure 9 Hamlonan Equaons of Moon (Chaper 8) Wha We Dd Las Tme Consruced Hamlonan formalsm Hqp (,,) = qp Lqq (,,) H p = q H q = p H L = Equvalen o Lagrangan formalsm Smpler, bu wce as

More information

Machine Learning 2nd Edition

Machine Learning 2nd Edition INTRODUCTION TO Lecure Sldes for Machne Learnng nd Edon ETHEM ALPAYDIN, modfed by Leonardo Bobadlla and some pars from hp://www.cs.au.ac.l/~aparzn/machnelearnng/ The MIT Press, 00 alpaydn@boun.edu.r hp://www.cmpe.boun.edu.r/~ehem/mle

More information

CH.3. COMPATIBILITY EQUATIONS. Continuum Mechanics Course (MMC) - ETSECCPB - UPC

CH.3. COMPATIBILITY EQUATIONS. Continuum Mechanics Course (MMC) - ETSECCPB - UPC CH.3. COMPATIBILITY EQUATIONS Connuum Mechancs Course (MMC) - ETSECCPB - UPC Overvew Compably Condons Compably Equaons of a Poenal Vecor Feld Compably Condons for Infnesmal Srans Inegraon of he Infnesmal

More information

THEORETICAL AUTOCORRELATIONS. ) if often denoted by γ. Note that

THEORETICAL AUTOCORRELATIONS. ) if often denoted by γ. Note that THEORETICAL AUTOCORRELATIONS Cov( y, y ) E( y E( y))( y E( y)) ρ = = Var( y) E( y E( y)) =,, L ρ = and Cov( y, y ) s ofen denoed by whle Var( y ) f ofen denoed by γ. Noe ha γ = γ and ρ = ρ and because

More information

Sampling Procedure of the Sum of two Binary Markov Process Realizations

Sampling Procedure of the Sum of two Binary Markov Process Realizations Samplng Procedure of he Sum of wo Bnary Markov Process Realzaons YURY GORITSKIY Dep. of Mahemacal Modelng of Moscow Power Insue (Techncal Unversy), Moscow, RUSSIA, E-mal: gorsky@yandex.ru VLADIMIR KAZAKOV

More information

Comparison of Differences between Power Means 1

Comparison of Differences between Power Means 1 In. Journal of Mah. Analyss, Vol. 7, 203, no., 5-55 Comparson of Dfferences beween Power Means Chang-An Tan, Guanghua Sh and Fe Zuo College of Mahemacs and Informaon Scence Henan Normal Unversy, 453007,

More information

Chapter 4. Neural Networks Based on Competition

Chapter 4. Neural Networks Based on Competition Chaper 4. Neural Neworks Based on Compeon Compeon s mporan for NN Compeon beween neurons has been observed n bologcal nerve sysems Compeon s mporan n solvng many problems To classfy an npu paern _1 no

More information

UNIVERSITAT AUTÒNOMA DE BARCELONA MARCH 2017 EXAMINATION

UNIVERSITAT AUTÒNOMA DE BARCELONA MARCH 2017 EXAMINATION INTERNATIONAL TRADE T. J. KEHOE UNIVERSITAT AUTÒNOMA DE BARCELONA MARCH 27 EXAMINATION Please answer wo of he hree quesons. You can consul class noes, workng papers, and arcles whle you are workng on he

More information

An introduction to Support Vector Machine

An introduction to Support Vector Machine An nroducon o Suppor Vecor Machne 報告者 : 黃立德 References: Smon Haykn, "Neural Neworks: a comprehensve foundaon, second edon, 999, Chaper 2,6 Nello Chrsann, John Shawe-Tayer, An Inroducon o Suppor Vecor Machnes,

More information

Bayes rule for a classification problem INF Discriminant functions for the normal density. Euclidean distance. Mahalanobis distance

Bayes rule for a classification problem INF Discriminant functions for the normal density. Euclidean distance. Mahalanobis distance INF 43 3.. Repeon Anne Solberg (anne@f.uo.no Bayes rule for a classfcaon problem Suppose we have J, =,...J classes. s he class label for a pxel, and x s he observed feaure vecor. We can use Bayes rule

More information

Time-interval analysis of β decay. V. Horvat and J. C. Hardy

Time-interval analysis of β decay. V. Horvat and J. C. Hardy Tme-nerval analyss of β decay V. Horva and J. C. Hardy Work on he even analyss of β decay [1] connued and resuled n he developmen of a novel mehod of bea-decay me-nerval analyss ha produces hghly accurae

More information

Pattern Classification (III) & Pattern Verification

Pattern Classification (III) & Pattern Verification Preare by Prof. Hu Jang CSE638 --4 CSE638 3. Seech & Language Processng o.5 Paern Classfcaon III & Paern Verfcaon Prof. Hu Jang Dearmen of Comuer Scence an Engneerng York Unversy Moel Parameer Esmaon Maxmum

More information

Anisotropic Behaviors and Its Application on Sheet Metal Stamping Processes

Anisotropic Behaviors and Its Application on Sheet Metal Stamping Processes Ansoropc Behavors and Is Applcaon on Shee Meal Sampng Processes Welong Hu ETA-Engneerng Technology Assocaes, Inc. 33 E. Maple oad, Sue 00 Troy, MI 48083 USA 48-79-300 whu@ea.com Jeanne He ETA-Engneerng

More information

General Weighted Majority, Online Learning as Online Optimization

General Weighted Majority, Online Learning as Online Optimization Sascal Technques n Robocs (16-831, F10) Lecure#10 (Thursday Sepember 23) General Weghed Majory, Onlne Learnng as Onlne Opmzaon Lecurer: Drew Bagnell Scrbe: Nahanel Barshay 1 1 Generalzed Weghed majory

More information

Implementation of Quantized State Systems in MATLAB/Simulink

Implementation of Quantized State Systems in MATLAB/Simulink SNE T ECHNICAL N OTE Implemenaon of Quanzed Sae Sysems n MATLAB/Smulnk Parck Grabher, Mahas Rößler 2*, Bernhard Henzl 3 Ins. of Analyss and Scenfc Compung, Venna Unversy of Technology, Wedner Haupsraße

More information

Approximate Analytic Solution of (2+1) - Dimensional Zakharov-Kuznetsov(Zk) Equations Using Homotopy

Approximate Analytic Solution of (2+1) - Dimensional Zakharov-Kuznetsov(Zk) Equations Using Homotopy Arcle Inernaonal Journal of Modern Mahemacal Scences, 4, (): - Inernaonal Journal of Modern Mahemacal Scences Journal homepage: www.modernscenfcpress.com/journals/jmms.aspx ISSN: 66-86X Florda, USA Approxmae

More information

CONSISTENT EARTHQUAKE ACCELERATION AND DISPLACEMENT RECORDS

CONSISTENT EARTHQUAKE ACCELERATION AND DISPLACEMENT RECORDS APPENDX J CONSSTENT EARTHQUAKE ACCEERATON AND DSPACEMENT RECORDS Earhqake Acceleraons can be Measred. However, Srcres are Sbjeced o Earhqake Dsplacemens J. NTRODUCTON { XE "Acceleraon Records" }A he presen

More information

PHYS 1443 Section 001 Lecture #4

PHYS 1443 Section 001 Lecture #4 PHYS 1443 Secon 001 Lecure #4 Monda, June 5, 006 Moon n Two Dmensons Moon under consan acceleraon Projecle Moon Mamum ranges and heghs Reerence Frames and relae moon Newon s Laws o Moon Force Newon s Law

More information

Introduction ( Week 1-2) Course introduction A brief introduction to molecular biology A brief introduction to sequence comparison Part I: Algorithms

Introduction ( Week 1-2) Course introduction A brief introduction to molecular biology A brief introduction to sequence comparison Part I: Algorithms Course organzaon Inroducon Wee -2) Course nroducon A bref nroducon o molecular bology A bref nroducon o sequence comparson Par I: Algorhms for Sequence Analyss Wee 3-8) Chaper -3, Models and heores» Probably

More information

We are estimating the density of long distant migrant (LDM) birds in wetlands along Lake Michigan.

We are estimating the density of long distant migrant (LDM) birds in wetlands along Lake Michigan. Ch 17 Random ffecs and Mxed Models 17. Random ffecs Models We are esmang he densy of long dsan mgran (LDM) brds n welands along Lake Mchgan. μ + = LDM per hecaren h weland ~ N(0, ) The varably of expeced

More information

Computing Relevance, Similarity: The Vector Space Model

Computing Relevance, Similarity: The Vector Space Model Compung Relevance, Smlary: The Vecor Space Model Based on Larson and Hears s sldes a UC-Bereley hp://.sms.bereley.edu/courses/s0/f00/ aabase Managemen Sysems, R. Ramarshnan ocumen Vecors v ocumens are

More information

Transcription: Messenger RNA, mrna, is produced and transported to Ribosomes

Transcription: Messenger RNA, mrna, is produced and transported to Ribosomes Quanave Cenral Dogma I Reference hp//book.bonumbers.org Inaon ranscrpon RNA polymerase and ranscrpon Facor (F) s bnds o promoer regon of DNA ranscrpon Meenger RNA, mrna, s produced and ranspored o Rbosomes

More information

Department of Economics University of Toronto

Department of Economics University of Toronto Deparmen of Economcs Unversy of Torono ECO408F M.A. Economercs Lecure Noes on Heeroskedascy Heeroskedascy o Ths lecure nvolves lookng a modfcaons we need o make o deal wh he regresson model when some of

More information

New M-Estimator Objective Function. in Simultaneous Equations Model. (A Comparative Study)

New M-Estimator Objective Function. in Simultaneous Equations Model. (A Comparative Study) Inernaonal Mahemacal Forum, Vol. 8, 3, no., 7 - HIKARI Ld, www.m-hkar.com hp://dx.do.org/.988/mf.3.3488 New M-Esmaor Objecve Funcon n Smulaneous Equaons Model (A Comparave Sudy) Ahmed H. Youssef Professor

More information

NPTEL Project. Econometric Modelling. Module23: Granger Causality Test. Lecture35: Granger Causality Test. Vinod Gupta School of Management

NPTEL Project. Econometric Modelling. Module23: Granger Causality Test. Lecture35: Granger Causality Test. Vinod Gupta School of Management P age NPTEL Proec Economerc Modellng Vnod Gua School of Managemen Module23: Granger Causaly Tes Lecure35: Granger Causaly Tes Rudra P. Pradhan Vnod Gua School of Managemen Indan Insue of Technology Kharagur,

More information

Chapters 2 Kinematics. Position, Distance, Displacement

Chapters 2 Kinematics. Position, Distance, Displacement Chapers Knemacs Poson, Dsance, Dsplacemen Mechancs: Knemacs and Dynamcs. Knemacs deals wh moon, bu s no concerned wh he cause o moon. Dynamcs deals wh he relaonshp beween orce and moon. The word dsplacemen

More information

TSS = SST + SSE An orthogonal partition of the total SS

TSS = SST + SSE An orthogonal partition of the total SS ANOVA: Topc 4. Orhogonal conrass [ST&D p. 183] H 0 : µ 1 = µ =... = µ H 1 : The mean of a leas one reamen group s dfferen To es hs hypohess, a basc ANOVA allocaes he varaon among reamen means (SST) equally

More information

Displacement, Velocity, and Acceleration. (WHERE and WHEN?)

Displacement, Velocity, and Acceleration. (WHERE and WHEN?) Dsplacemen, Velocy, and Acceleraon (WHERE and WHEN?) Mah resources Append A n your book! Symbols and meanng Algebra Geomery (olumes, ec.) Trgonomery Append A Logarhms Remnder You wll do well n hs class

More information

, t 1. Transitions - this one was easy, but in general the hardest part is choosing the which variables are state and control variables

, t 1. Transitions - this one was easy, but in general the hardest part is choosing the which variables are state and control variables Opmal Conrol Why Use I - verss calcls of varaons, opmal conrol More generaly More convenen wh consrans (e.g., can p consrans on he dervaves More nsghs no problem (a leas more apparen han hrogh calcls of

More information

FTCS Solution to the Heat Equation

FTCS Solution to the Heat Equation FTCS Soluon o he Hea Equaon ME 448/548 Noes Gerald Reckenwald Porland Sae Unversy Deparmen of Mechancal Engneerng gerry@pdxedu ME 448/548: FTCS Soluon o he Hea Equaon Overvew Use he forward fne d erence

More information

On One Analytic Method of. Constructing Program Controls

On One Analytic Method of. Constructing Program Controls Appled Mahemacal Scences, Vol. 9, 05, no. 8, 409-407 HIKARI Ld, www.m-hkar.com hp://dx.do.org/0.988/ams.05.54349 On One Analyc Mehod of Consrucng Program Conrols A. N. Kvko, S. V. Chsyakov and Yu. E. Balyna

More information

arxiv: v1 [math.oc] 11 Dec 2014

arxiv: v1 [math.oc] 11 Dec 2014 Nework Newon Aryan Mokhar, Qng Lng and Alejandro Rbero Dep. of Elecrcal and Sysems Engneerng, Unversy of Pennsylvana Dep. of Auomaon, Unversy of Scence and Technology of Chna arxv:1412.374v1 [mah.oc] 11

More information

FI 3103 Quantum Physics

FI 3103 Quantum Physics /9/4 FI 33 Quanum Physcs Aleander A. Iskandar Physcs of Magnesm and Phooncs Research Grou Insu Teknolog Bandung Basc Conces n Quanum Physcs Probably and Eecaon Value Hesenberg Uncerany Prncle Wave Funcon

More information

ABSTRACT KEYWORDS. Bonus-malus systems, frequency component, severity component. 1. INTRODUCTION

ABSTRACT KEYWORDS. Bonus-malus systems, frequency component, severity component. 1. INTRODUCTION EERAIED BU-MAU YTEM ITH A FREQUECY AD A EVERITY CMET A IDIVIDUA BAI I AUTMBIE IURACE* BY RAHIM MAHMUDVAD AD HEI HAAI ABTRACT Frangos and Vronos (2001) proposed an opmal bonus-malus sysems wh a frequency

More information

This document is downloaded from DR-NTU, Nanyang Technological University Library, Singapore.

This document is downloaded from DR-NTU, Nanyang Technological University Library, Singapore. Ths documen s downloaded from DR-NTU, Nanyang Technologcal Unversy Lbrary, Sngapore. Tle A smplfed verb machng algorhm for word paron n vsual speech processng( Acceped verson ) Auhor(s) Foo, Say We; Yong,

More information

An Effective TCM-KNN Scheme for High-Speed Network Anomaly Detection

An Effective TCM-KNN Scheme for High-Speed Network Anomaly Detection Vol. 24, November,, 200 An Effecve TCM-KNN Scheme for Hgh-Speed Nework Anomaly eecon Yang L Chnese Academy of Scences, Bejng Chna, 00080 lyang@sofware.c.ac.cn Absrac. Nework anomaly deecon has been a ho

More information

Response of MDOF systems

Response of MDOF systems Response of MDOF syses Degree of freedo DOF: he nu nuber of ndependen coordnaes requred o deerne copleely he posons of all pars of a syse a any nsan of e. wo DOF syses hree DOF syses he noral ode analyss

More information

Approximation Lasso Methods for Language Modeling

Approximation Lasso Methods for Language Modeling Approxmaon Lasso Mehods for Language Modelng Janfeng Gao Mcrosof Research One Mcrosof Way Redmond WA 98052 USA jfgao@mcrosof.com Hsam Suzuk Mcrosof Research One Mcrosof Way Redmond WA 98052 USA hsams@mcrosof.com

More information

How about the more general "linear" scalar functions of scalars (i.e., a 1st degree polynomial of the following form with a constant term )?

How about the more general linear scalar functions of scalars (i.e., a 1st degree polynomial of the following form with a constant term )? lmcd Lnear ransformaon of a vecor he deas presened here are que general hey go beyond he radonal mar-vecor ype seen n lnear algebra Furhermore, hey do no deal wh bass and are equally vald for any se of

More information

Motion in Two Dimensions

Motion in Two Dimensions Phys 1 Chaper 4 Moon n Two Dmensons adzyubenko@csub.edu hp://www.csub.edu/~adzyubenko 005, 014 A. Dzyubenko 004 Brooks/Cole 1 Dsplacemen as a Vecor The poson of an objec s descrbed by s poson ecor, r The

More information

MANY real-world applications (e.g. production

MANY real-world applications (e.g. production Barebones Parcle Swarm for Ineger Programmng Problems Mahamed G. H. Omran, Andres Engelbrech and Ayed Salman Absrac The performance of wo recen varans of Parcle Swarm Opmzaon (PSO) when appled o Ineger

More information

Density Matrix Description of NMR BCMB/CHEM 8190

Density Matrix Description of NMR BCMB/CHEM 8190 Densy Marx Descrpon of NMR BCMBCHEM 89 Operaors n Marx Noaon If we say wh one bass se, properes vary only because of changes n he coeffcens weghng each bass se funcon x = h< Ix > - hs s how we calculae

More information

Notes on the stability of dynamic systems and the use of Eigen Values.

Notes on the stability of dynamic systems and the use of Eigen Values. Noes on he sabl of dnamc ssems and he use of Egen Values. Source: Macro II course noes, Dr. Davd Bessler s Tme Seres course noes, zarads (999) Ineremporal Macroeconomcs chaper 4 & Techncal ppend, and Hamlon

More information

Hidden Markov Models

Hidden Markov Models 11-755 Machne Learnng for Sgnal Processng Hdden Markov Models Class 15. 12 Oc 2010 1 Admnsrva HW2 due Tuesday Is everyone on he projecs page? Where are your projec proposals? 2 Recap: Wha s an HMM Probablsc

More information

First-order piecewise-linear dynamic circuits

First-order piecewise-linear dynamic circuits Frs-order pecewse-lnear dynamc crcus. Fndng he soluon We wll sudy rs-order dynamc crcus composed o a nonlnear resse one-por, ermnaed eher by a lnear capacor or a lnear nducor (see Fg.. Nonlnear resse one-por

More information

Lecture 18: The Laplace Transform (See Sections and 14.7 in Boas)

Lecture 18: The Laplace Transform (See Sections and 14.7 in Boas) Lecure 8: The Lalace Transform (See Secons 88- and 47 n Boas) Recall ha our bg-cure goal s he analyss of he dfferenal equaon, ax bx cx F, where we emloy varous exansons for he drvng funcon F deendng on

More information

Improved Classification Based on Predictive Association Rules

Improved Classification Based on Predictive Association Rules Proceedngs of he 009 IEEE Inernaonal Conference on Sysems, Man, and Cybernecs San Anono, TX, USA - Ocober 009 Improved Classfcaon Based on Predcve Assocaon Rules Zhxn Hao, Xuan Wang, Ln Yao, Yaoyun Zhang

More information

Density Matrix Description of NMR BCMB/CHEM 8190

Density Matrix Description of NMR BCMB/CHEM 8190 Densy Marx Descrpon of NMR BCMBCHEM 89 Operaors n Marx Noaon Alernae approach o second order specra: ask abou x magnezaon nsead of energes and ranson probables. If we say wh one bass se, properes vary

More information

. The geometric multiplicity is dim[ker( λi. number of linearly independent eigenvectors associated with this eigenvalue.

. The geometric multiplicity is dim[ker( λi. number of linearly independent eigenvectors associated with this eigenvalue. Lnear Algebra Lecure # Noes We connue wh he dscusson of egenvalues, egenvecors, and dagonalzably of marces We wan o know, n parcular wha condons wll assure ha a marx can be dagonalzed and wha he obsrucons

More information

Example: MOSFET Amplifier Distortion

Example: MOSFET Amplifier Distortion 4/25/2011 Example MSFET Amplfer Dsoron 1/9 Example: MSFET Amplfer Dsoron Recall hs crcu from a prevous handou: ( ) = I ( ) D D d 15.0 V RD = 5K v ( ) = V v ( ) D o v( ) - K = 2 0.25 ma/v V = 2.0 V 40V.

More information

CHAPTER 2: Supervised Learning

CHAPTER 2: Supervised Learning HATER 2: Supervsed Learnng Learnng a lass from Eamples lass of a famly car redcon: Is car a famly car? Knowledge eracon: Wha do people epec from a famly car? Oupu: osve (+) and negave ( ) eamples Inpu

More information

A Generalized Online Mirror Descent with Applications to Classification and Regression

A Generalized Online Mirror Descent with Applications to Classification and Regression Journal of Machne Learnng Research 1 000 1-48 Submed 4/00; Publshed 10/00 A Generalzed Onlne Mrror Descen wh Applcaons o Classfcaon and Regresson Francesco Orabona Toyoa Technologcal Insue a Chcago 60637

More information

Second-Order Non-Stationary Online Learning for Regression

Second-Order Non-Stationary Online Learning for Regression Second-Order Non-Saonary Onlne Learnng for Regresson Nna Vas, Edward Moroshko, and Koby Crammer, Fellow, IEEE arxv:303040v cslg] Mar 03 Absrac he goal of a learner, n sandard onlne learnng, s o have he

More information

by Lauren DeDieu Advisor: George Chen

by Lauren DeDieu Advisor: George Chen b Laren DeDe Advsor: George Chen Are one of he mos powerfl mehods o nmercall solve me dependen paral dfferenal eqaons PDE wh some knd of snglar shock waves & blow-p problems. Fed nmber of mesh pons Moves

More information

CS286.2 Lecture 14: Quantum de Finetti Theorems II

CS286.2 Lecture 14: Quantum de Finetti Theorems II CS286.2 Lecure 14: Quanum de Fne Theorems II Scrbe: Mara Okounkova 1 Saemen of he heorem Recall he las saemen of he quanum de Fne heorem from he prevous lecure. Theorem 1 Quanum de Fne). Le ρ Dens C 2

More information

2 Aggregate demand in partial equilibrium static framework

2 Aggregate demand in partial equilibrium static framework Unversy of Mnnesoa 8107 Macroeconomc Theory, Sprng 2009, Mn 1 Fabrzo Perr Lecure 1. Aggregaon 1 Inroducon Probably so far n he macro sequence you have deal drecly wh represenave consumers and represenave

More information

Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation

Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation Course Noes for EE7C Spring 018: Convex Opimizaion and Approximaion Insrucor: Moriz Hard Email: hard+ee7c@berkeley.edu Graduae Insrucor: Max Simchowiz Email: msimchow+ee7c@berkeley.edu Ocober 15, 018 3

More information