Supplement for SADAGRAD: Strongly Adaptive Stochastic Gradient Methods"
|
|
- August Williamson
- 5 years ago
- Views:
Transcription
1 Suppleme for SADAGRAD: Srogly Adapive Sochasic Gradie Mehods" Zaiyi Che * 1 Yi Xu * Ehog Che 1 iabao Yag 1. Proof of Proposiio 1 Proposiio 1. Le ɛ > 0 be fixed, H 0 γi, γ g, EF (w 1 ) F (w ) ɛ 0 ad ieraio umber saisfies ɛ ɛ0(γ i g 1:,i ) η, η d g 1:,i. Algorihm 1 gives a soluio ŵ such ha EF (ŵ ) F ɛ. Proof. Le ψ 0 (w) 0 ad x H x Hx. Firs, we ca see ha ψ 1 (w) ψ (w) for ay 0. Defie z τ1 g ad τ ( F (w ) g ) (w w). Le ψ be defied by ψ (g) sup g x 1 x Ω η ψ (x) aig he summaio of objecive gap i all ieraios, we (F (w ) F (w)) g (w w) g w 1 η ψ (w) sup x Ω F (w ) (w w) g w 1 η ψ (w) 1 η ψ (w) g w g x 1 η ψ (x) 1 η ψ (w) g w ψ ( z ) Noe ha ψ ( z ) g w 1 1 η ψ (w 1 ) g w 1 1 η ψ 1(w 1 ) sup z x 1 x Ω η ψ 1(x) ψ 1( z ) ψ 1( z 1 ) g ψ 1( z 1 ) η g ψ 1 where he las iequaliy uses he fac ha ψ (w) is 1- srogly covex w.r. ψ H ad cosequeially ψ (w) is η-smooh wr.. ψ H 1. hus, we g w ψ ( z ) g w ψ 1( z 1 ) g ψ 1( z 1 ) η g ψ 1 1 g w ψ 1( z 1 ) η g ψ 1 By repeaig his process, we g w ψ ( z ) ψ 0( z 0 ) η η g ψ 1 g ψ 1 he (F (w ) F (w)) 1 η ψ (w) η g ψ 1 (1)
2 SADAGRAD: Srogly Adapive Sochasic Gradie Mehods Followig he aalysis i (Duchi e al., 011), we hus g ψ g 1 1:,i (F (w ) F (w)) γ w w 1 (w w 1) diag(s )(w w 1 ) η η η g 1:,i γ i g 1:,i w w 1 η η g 1:,i Now by he value of ɛ ɛ0(γ i g 1:,i ) η, η d g 1:,i, we (γ i g 1:,i ) η η d g 1:,i ɛ ɛ 4ɛ 0 Dividig by o boh sides ad seig w w, followig he iequaliy () ad he covexiy of F (w) we F (ŵ) F ɛ 4ɛ 0 w w 1 ɛ 1 Le F be he filraio associaed wih Algorihm 1 i he paper. Noicig ha is a radom variable wih respec o F, we cao ge rid of he las erm direcly. Defie he Sequece X N as X 1 i 1 g i Eg i, w i w () where Eg i F (w i ). ( Sice E g 1 Eg 1 0 ad ) w 1 arg mi w Ω ηw 1 τ1 g τ 1 ψ (w), which is measurable wih respec o g 1,..., g ad w 1,..., w, i is easy o see N is a marigale differece sequece wih respec o F, e.g. E F 1 0. O he oher had, sice g is upper bouded (e.g., by G), followig he saeme of i he heorem, N 4 ɛ ( Gɛ0 θ ), θ d G < always holds. he followig Lemma 1 below we ha EX 0. Now aig he expecaio we ha EF (ŵ) F ɛ E w w 1 ɛ 4ɛ E 1 0 ɛ E F (w 1 ) F (w ) ɛ ɛ 0 0 ɛ he we fiish he proof. Lemma 1. Le N be a marigale differece sequece w.r. he filraio F N, is a soppig ime such ha F for all N. If 0 < N <, he we Proof. 1 E 1 E 0. E 1 I( ) E F N I( )E 1 F I( ) E 0 I( ) I( ) I( ) I( ) E F F F E F E E F F 1 E F 1 where I( ) is he idicaor fucio. he firs equaio follows from he defiiio of codiioal expecaio ad N; he secod equaio follows from he fac ha I( ) 1; he hird ad fifh equaios follow from he defiiio of soppig ime (( ) F ); he seveh ad las equaios follow from he defiiio of marigale differece sequece; ad eighh equaio follows from heorem i (Durre, 010).
3 SADAGRAD: Srogly Adapive Sochasic Gradie Mehods. Proof of heorem 1 heorem 1. Cosider SCO (1) wih a propery () ad a give ɛ > 0. Assume H 0 γi i Algorihm 1 ad γ,τ gτ, F (w 0 ) F ɛ 0 ad is he miimum umber such ha ɛ (γ i g 1:,i ) θ, θ d g 1:,i K log (ɛ 0 /ɛ), we EF (w K ) F ɛ.. Wih Proof of heorem 1. We will show by iducio ha EF (w ) F ɛ ɛ0 for 0, 1,..., K, which leads o our coclusio whe K log(ɛ 0 /ɛ). he iequaliy holds obviously for 0. Codiioed o EF (w 1 ) F ɛ 1, we will show ha EF (w ) F ɛ. We will modify Proposiio 1, he use i o he -h epoch of Algorihm codiioed o radomess i previous epoches. Le E deoes he expecaio over all radomess before he las ieraio of he -h epoch ad E 1: 1 deoes he expecaio over he radomess i he -h epoch give he radomess before -h epoch. Give w 1, we le w 1 deoe he opimal soluio ha is closes o w 1 1. Accordig o he proof of Proposio 1, We E 1: 1 F (w ) F (w 1) γ i g1: E,i 1: 1 w 1 w η 1 η d g 1:,i Eg g, w w 1 By he value of η θ ɛ / ad 4(γ i g 1:,i ), θ d g 1:,i ɛ, we hus θ ɛ (γ i g 1:,i ) η 8 η d g 1:,i ɛ E 1: 1 F (w ) F (w 1) E 1: 1 8 w 1 w 1 ɛ Eg g, w w 1 1 Sice we oly assume he codiio () ha does o ecessarily imply he uiqueess of he opimal soluios. he followig he similar argumes i Proposiio 1, we E 1: 1 F (w ) F (w 1) E 1: 1 8 w 1 w 1 ɛ aig expecaio over radomess i sages 1,..., 1, we EF (w ) F (w 1) E 8 w 1 w 1 ɛ 1 4 EF (w 1) F ɛ ɛ 1 4 ɛ ɛ herefore by iducio, we EF (w K ) F ɛ K ɛ.. Proof of heorem Lemma. Cosider SCO (4) wih he propery (). Le H 0 γi i Algorihm ad γ g. For ay w Ω ad is closes opimal soluio w, we F ( w ) F (w) G w 1 w 1 1 (Eg g ) (w w) η d g 1:,i γ i g 1:,i w w 1 η where w 1 w /. Proof. his proof is similar o he proof of Proposiio 1, bu we do o ae expecaio here. For compleeess, we give he proof here. hroughou he whole proof, we se he oaio g as he sochasic gradie of f(w ) ad as a resul Eg f(w ). Le ψ 0 (w) 0 ad x H x Hx. Firs, we ca see ha ψ 1 (w) ψ (w) for ay 0. Defie z τ1 g ad τ ( f(w ) g ) (w w). Le ψ be defied by ψ (g) sup g x 1 x Ω η ψ (x) φ(x) aig he summaio of objecive gap i all ieraios, we
4 SADAGRAD: Srogly Adapive Sochasic Gradie Mehods ψ (w) is η-smooh w.r.. ψ H 1. hus, we (f(w ) f(w) φ(w ) φ(w)) ( f(w ) (w w) φ(w ) φ(w)) g (w w) g w 1 η ψ (w) 1 η ψ (w) sup x Ω (φ(w ) φ(w)) g w 1 η ψ (w) φ(w) φ(w ) g w φ(w ) g x 1 η ψ (x) φ(x) 1 η ψ (w) g w φ(w ) ψ ( z ) () Noe ha ψ ( z ) g w 1 1 η ψ (w 1 ) φ(w 1 ) g w 1 1 η ψ 1(w 1 ) ( 1)φ(w 1 ) φ(w 1 ) sup z x 1 x Ω η ψ 1(x) ( 1)φ(x) φ(w 1 ) ψ 1( z ) φ(w 1 ) ψ 1( z 1 ) g ψ 1( z 1 ) η g ψ 1 φ(w 1 ) where he las iequaliy uses he fac ha ψ (w) is 1- srogly covex w.r. ψ H ad cosequeially g w ψ ( z ) g w ψ 1( z 1 ) g ψ 1( z 1 ) η g ψ 1 φ(w 1) 1 g w ψ 1( z 1 ) η g ψ 1 φ(w 1 ) By repeaig his process, we g w ψ ( z ) ψ 0( z 0 ) η η g ψ φ(w 1 1 ) g ψ φ(w 1 1 ) (4) Pluggig iequaliy (4) i iequaliy (), he (F (w ) F (w)) 1 η ψ (w) η φ(w 1 ) g ψ 1 φ(w 1 ) By addig F (w 1 ) F (w 1 ) o he boh sides of above iequaliy ad usig he fac ha F (w) f(w) φ(w), we ge 1 (F (w ) F (w)) 1 η ψ (w) η f(w 1 ) g ψ 1 f(w 1 ) Followig he aalysis i (Duchi e al., 011), we g ψ g 1 1:,i
5 SADAGRAD: Srogly Adapive Sochasic Gradie Mehods hus 1 (F (w ) F (w)) γ w w 1 (w w 1) diag(s )(w w 1 ) η η η g 1:,i f(w 1 ) f(w 1 ) γ i g 1:,i w w 1 η η g 1:,i ( f(w 1 )) (w 1 w 1 ) γ i g 1:,i w w 1 η η G w 1 w 1 g 1:,i where he las iequaliy hold usig Cauchy-Schwarz Iequaliy ad he fac ha f(w 1 ) G. Dividig by o boh sides, he we fiish he proof by usig he covexiy of F (w). heorem. For a give ɛ > 0, le K log (ɛ 0 /ɛ). Assume H 0 γi ad γ,τ gτ, F (w 0 ) F ɛ 0 ad is he miimum umber such G w 1 ɛ A, w 1 ɛ, where ha A (γ i g 1:,i ) θ, θ d g 1:,i. Algorihm 4 guaraees ha EF (w K ) F ɛ. Proof. his resul is proved by revisig Lemma o hold for a bouded soppig ime of he supermarigale sequece X i (). aig he expecaio of Lemma, we ha G w1 w 1 EF ( w ) F (w) E 1 E (Eg g ) (w w) η d E g 1:,i γ i g 1:,i w w 1 η he followig he same argumes o Proposiio 1, we ha E 1 (Eg g ) (w w) 0 Similar o he iducio of heorem 1, le η θ ɛ / ad he ieraio umber i -h epoch o be he smalles umber saisfyig followig iequaliies (γ i g 1:,i ) η 1 η d g 1:,i G w 1 w 1 ɛ ɛ hus codiioed o 1,..., 1-h epoches, we ha E 1: 1 F (w ) F (w 1) E 1: 1 1 w 1 w 1 ɛ aig expecaio over radomess i sages 1,..., 1, we EF (w ) F (w 1) E 1 w 1 w 1 ɛ 1 6 EF (w 1) F ɛ ɛ 1 6 ɛ ɛ herefore by iducio, we EF (w K ) F ɛ K ɛ. 4. Proof of heorem heorem. Uder he same assumpios as heorem 1 ad F (w 0 ) F ɛ 0, where w 0 is a iiial soluio. Le 1, ɛ ɛ0, K log ɛ0 ɛ ad (s) (γ i g sɛ 1:,i ) θ, θ d g 1:,i. he wih a mos a oal umber of S log ( 1 ) 1 calls of SADAGRAD ad a worse-cas ieraio complexiy of O(1/(ɛ)), Algorihm 5 fids a soluio w (S) such ha EF (w (S) ) F ɛ. Proof. Sice 1 / > 1, he F (w 0 ) F ( 1 /)ɛ 0. Followig he proof of heorem 1, we ca show ha EF (w (1) ) F ( ( ) 1/)ɛ 0 1 K ɛ ɛ wih K log 0ɛ ad (1) (γ i g 1:,i ) θ, θ d g 1:,i ( 1 ɛ ),
6 1,..., K. Nex, sice ɛ ɛ0, he we EF (w (1) ) F ( 1 ) ɛ0 ( ) ɛ0. By ruig SADAGRAD from w (1), heorem 1 esures ha EF (w () ) F EF (w(1) ) F K ( /)ɛ 0 ( ) K ɛ By coiuig he process, wih S ( log 1 ) 1, we ( ) EF (w (S) S ) F ɛ ɛ (5) he oal umber of ieraios for he S calls of SADAGRAD is upper bouded by oal for some C > 0. S K (s) s1 1 Acowledgeme SADAGRAD: Srogly Adapive Sochasic Gradie Mehods S s1 C S K s 1 1 ɛ 0 s1 1 ( ) 1 O ɛ C s ɛ 0 1 K 1 1 We ha Prof. Qihe ag from Uiversiy of Iowa for his help o he proof of Lemma 1. Refereces Duchi, Joh, Haza, Elad, ad Siger, Yoram. Adapive subgradie mehods for olie learig ad sochasic opimizaio. Joural of Machie Learig Research, 1 (Jul):11 159, 011. Durre, Ric. Probabiliy: heory ad examples. Cambridge uiversiy press, 010.
1 Notes on Little s Law (l = λw)
Copyrigh c 26 by Karl Sigma Noes o Lile s Law (l λw) We cosider here a famous ad very useful law i queueig heory called Lile s Law, also kow as l λw, which assers ha he ime average umber of cusomers i
More informationLecture 15 First Properties of the Brownian Motion
Lecure 15: Firs Properies 1 of 8 Course: Theory of Probabiliy II Term: Sprig 2015 Isrucor: Gorda Zikovic Lecure 15 Firs Properies of he Browia Moio This lecure deals wih some of he more immediae properies
More informationExtremal graph theory II: K t and K t,t
Exremal graph heory II: K ad K, Lecure Graph Theory 06 EPFL Frak de Zeeuw I his lecure, we geeralize he wo mai heorems from he las lecure, from riagles K 3 o complee graphs K, ad from squares K, o complee
More informationSome Properties of Semi-E-Convex Function and Semi-E-Convex Programming*
The Eighh Ieraioal Symposium o Operaios esearch ad Is Applicaios (ISOA 9) Zhagjiajie Chia Sepember 2 22 29 Copyrigh 29 OSC & APOC pp 33 39 Some Properies of Semi-E-Covex Fucio ad Semi-E-Covex Programmig*
More informationActuarial Society of India
Acuarial Sociey of Idia EXAMINAIONS Jue 5 C4 (3) Models oal Marks - 5 Idicaive Soluio Q. (i) a) Le U deoe he process described by 3 ad V deoe he process described by 4. he 5 e 5 PU [ ] PV [ ] ( e ).538!
More informationMath 6710, Fall 2016 Final Exam Solutions
Mah 67, Fall 6 Fial Exam Soluios. Firs, a sude poied ou a suble hig: if P (X i p >, he X + + X (X + + X / ( evaluaes o / wih probabiliy p >. This is roublesome because a radom variable is supposed o be
More informationLecture 9: Polynomial Approximations
CS 70: Complexiy Theory /6/009 Lecure 9: Polyomial Approximaios Isrucor: Dieer va Melkebeek Scribe: Phil Rydzewski & Piramaayagam Arumuga Naiar Las ime, we proved ha o cosa deph circui ca evaluae he pariy
More informationFIXED FUZZY POINT THEOREMS IN FUZZY METRIC SPACE
Mohia & Samaa, Vol. 1, No. II, December, 016, pp 34-49. ORIGINAL RESEARCH ARTICLE OPEN ACCESS FIED FUZZY POINT THEOREMS IN FUZZY METRIC SPACE 1 Mohia S. *, Samaa T. K. 1 Deparme of Mahemaics, Sudhir Memorial
More informationMathematical Statistics. 1 Introduction to the materials to be covered in this course
Mahemaical Saisics Iroducio o he maerials o be covered i his course. Uivariae & Mulivariae r.v s 2. Borl-Caelli Lemma Large Deviaios. e.g. X,, X are iid r.v s, P ( X + + X where I(A) is a umber depedig
More informationOnline Supplement to Reactive Tabu Search in a Team-Learning Problem
Olie Suppleme o Reacive abu Search i a eam-learig Problem Yueli She School of Ieraioal Busiess Admiisraio, Shaghai Uiversiy of Fiace ad Ecoomics, Shaghai 00433, People s Republic of Chia, she.yueli@mail.shufe.edu.c
More informationSupplementary Material for Fast Stochastic AUC Maximization with O(1/n)-Convergence Rate
Supplemetary Material for Fast Stochastic AUC Maximizatio with O/-Covergece Rate Migrui Liu Xiaoxua Zhag Zaiyi Che Xiaoyu Wag 3 iabao Yag echical Lemmas ized versio of Hoeffdig s iequality, ote that We
More informationBEST LINEAR FORECASTS VS. BEST POSSIBLE FORECASTS
BEST LINEAR FORECASTS VS. BEST POSSIBLE FORECASTS Opimal ear Forecasig Alhough we have o meioed hem explicily so far i he course, here are geeral saisical priciples for derivig he bes liear forecas, ad
More informationSupplementary Material for Fast Stochastic AUC Maximization with O(1/n)-Convergence Rate
Supplemetary Material for Fast Stochastic AUC Maximizatio with O/-Covergece Rate Migrui Liu Xiaoxua Zhag Zaiyi Che Xiaoyu Wag 3 iabao Yag echical Lemmas ized versio of Hoeffdig s iequality, ote that We
More informationMATH 507a ASSIGNMENT 4 SOLUTIONS FALL 2018 Prof. Alexander. g (x) dx = g(b) g(0) = g(b),
MATH 57a ASSIGNMENT 4 SOLUTIONS FALL 28 Prof. Alexader (2.3.8)(a) Le g(x) = x/( + x) for x. The g (x) = /( + x) 2 is decreasig, so for a, b, g(a + b) g(a) = a+b a g (x) dx b so g(a + b) g(a) + g(b). Sice
More informationAdditional Tables of Simulation Results
Saisica Siica: Suppleme REGULARIZING LASSO: A CONSISTENT VARIABLE SELECTION METHOD Quefeg Li ad Ju Shao Uiversiy of Wiscosi, Madiso, Eas Chia Normal Uiversiy ad Uiversiy of Wiscosi, Madiso Supplemeary
More informationReview Answers for E&CE 700T02
Review Aswers for E&CE 700T0 . Deermie he curre soluio, all possible direcios, ad sepsizes wheher improvig or o for he simple able below: 4 b ma c 0 0 0-4 6 0 - B N B N ^0 0 0 curre sol =, = Ch for - -
More informationSTK4080/9080 Survival and event history analysis
STK48/98 Survival ad eve hisory aalysis Marigales i discree ime Cosider a sochasic process The process M is a marigale if Lecure 3: Marigales ad oher sochasic processes i discree ime (recap) where (formally
More informationCOS 522: Complexity Theory : Boaz Barak Handout 10: Parallel Repetition Lemma
COS 522: Complexiy Theory : Boaz Barak Hadou 0: Parallel Repeiio Lemma Readig: () A Parallel Repeiio Theorem / Ra Raz (available o his websie) (2) Parallel Repeiio: Simplificaios ad he No-Sigallig Case
More informationTAKA KUSANO. laculty of Science Hrosh tlnlersty 1982) (n-l) + + Pn(t)x 0, (n-l) + + Pn(t)Y f(t,y), XR R are continuous functions.
Iera. J. Mah. & Mah. Si. Vol. 6 No. 3 (1983) 559-566 559 ASYMPTOTIC RELATIOHIPS BETWEEN TWO HIGHER ORDER ORDINARY DIFFERENTIAL EQUATIONS TAKA KUSANO laculy of Sciece Hrosh llersy 1982) ABSTRACT. Some asympoic
More informationL-functions and Class Numbers
L-fucios ad Class Numbers Sude Number Theory Semiar S. M.-C. 4 Sepember 05 We follow Romyar Sharifi s Noes o Iwasawa Theory, wih some help from Neukirch s Algebraic Number Theory. L-fucios of Dirichle
More informationComparison between Fourier and Corrected Fourier Series Methods
Malaysia Joural of Mahemaical Scieces 7(): 73-8 (13) MALAYSIAN JOURNAL OF MATHEMATICAL SCIENCES Joural homepage: hp://eispem.upm.edu.my/oural Compariso bewee Fourier ad Correced Fourier Series Mehods 1
More informationLecture 8 April 18, 2018
Sas 300C: Theory of Saisics Sprig 2018 Lecure 8 April 18, 2018 Prof Emmauel Cades Scribe: Emmauel Cades Oulie Ageda: Muliple Tesig Problems 1 Empirical Process Viewpoi of BHq 2 Empirical Process Viewpoi
More informationA note on deviation inequalities on {0, 1} n. by Julio Bernués*
A oe o deviaio iequaliies o {0, 1}. by Julio Berués* Deparameo de Maemáicas. Faculad de Ciecias Uiversidad de Zaragoza 50009-Zaragoza (Spai) I. Iroducio. Le f: (Ω, Σ, ) IR be a radom variable. Roughly
More informationAn interesting result about subset sums. Nitu Kitchloo. Lior Pachter. November 27, Abstract
A ieresig resul abou subse sums Niu Kichloo Lior Pacher November 27, 1993 Absrac We cosider he problem of deermiig he umber of subses B f1; 2; : : :; g such ha P b2b b k mod, where k is a residue class
More informationNEWTON METHOD FOR DETERMINING THE OPTIMAL REPLENISHMENT POLICY FOR EPQ MODEL WITH PRESENT VALUE
Yugoslav Joural of Operaios Research 8 (2008, Number, 53-6 DOI: 02298/YUJOR080053W NEWTON METHOD FOR DETERMINING THE OPTIMAL REPLENISHMENT POLICY FOR EPQ MODEL WITH PRESENT VALUE Jeff Kuo-Jug WU, Hsui-Li
More informationThe Connection between the Basel Problem and a Special Integral
Applied Mahemaics 4 5 57-584 Published Olie Sepember 4 i SciRes hp://wwwscirporg/joural/am hp://ddoiorg/436/am45646 The Coecio bewee he Basel Problem ad a Special Iegral Haifeg Xu Jiuru Zhou School of
More informationAn approximate approach to the exponential utility indifference valuation
A approximae approach o he expoeial uiliy idifferece valuaio akuji Arai Faculy of Ecoomics, Keio Uiversiy, 2-15-45 Mia, Miao-ku, okyo, 18-8345, Japa e-mail: arai@ecokeioacjp) Absrac We propose, i his paper,
More informationCalculus Limits. Limit of a function.. 1. One-Sided Limits...1. Infinite limits 2. Vertical Asymptotes...3. Calculating Limits Using the Limit Laws.
Limi of a fucio.. Oe-Sided..... Ifiie limis Verical Asympoes... Calculaig Usig he Limi Laws.5 The Squeeze Theorem.6 The Precise Defiiio of a Limi......7 Coiuiy.8 Iermediae Value Theorem..9 Refereces..
More informationFermat Numbers in Multinomial Coefficients
1 3 47 6 3 11 Joural of Ieger Sequeces, Vol. 17 (014, Aricle 14.3. Ferma Numbers i Muliomial Coefficies Shae Cher Deparme of Mahemaics Zhejiag Uiversiy Hagzhou, 31007 Chia chexiaohag9@gmail.com Absrac
More informationMoment Generating Function
1 Mome Geeraig Fucio m h mome m m m E[ ] x f ( x) dx m h ceral mome m m m E[( ) ] ( ) ( x ) f ( x) dx Mome Geeraig Fucio For a real, M () E[ e ] e k x k e p ( x ) discree x k e f ( x) dx coiuous Example
More informationResearch Article A Generalized Nonlinear Sum-Difference Inequality of Product Form
Joural of Applied Mahemaics Volume 03, Aricle ID 47585, 7 pages hp://dx.doi.org/0.55/03/47585 Research Aricle A Geeralized Noliear Sum-Differece Iequaliy of Produc Form YogZhou Qi ad Wu-Sheg Wag School
More informationThe Central Limit Theorem
The Ceral Limi Theorem The ceral i heorem is oe of he mos impora heorems i probabiliy heory. While here a variey of forms of he ceral i heorem, he mos geeral form saes ha give a sufficiely large umber,
More informationarxiv: v1 [math.pr] 16 Dec 2018
218, 1 17 () arxiv:1812.7383v1 [mah.pr] 16 Dec 218 Refleced BSDEs wih wo compleely separaed barriers ad regulaed rajecories i geeral filraio. Baadi Brahim ad Oukie Youssef Ib Tofaïl Uiversiy, Deparme of
More informationA Note on Prediction with Misspecified Models
ITB J. Sci., Vol. 44 A, No. 3,, 7-9 7 A Noe o Predicio wih Misspecified Models Khresha Syuhada Saisics Research Divisio, Faculy of Mahemaics ad Naural Scieces, Isiu Tekologi Badug, Jala Gaesa Badug, Jawa
More informationBE.430 Tutorial: Linear Operator Theory and Eigenfunction Expansion
BE.43 Tuorial: Liear Operaor Theory ad Eigefucio Expasio (adaped fro Douglas Lauffeburger) 9//4 Moivaig proble I class, we ecouered parial differeial equaios describig rasie syses wih cheical diffusio.
More informationBig O Notation for Time Complexity of Algorithms
BRONX COMMUNITY COLLEGE of he Ciy Uiversiy of New York DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE CSI 33 Secio E01 Hadou 1 Fall 2014 Sepember 3, 2014 Big O Noaio for Time Complexiy of Algorihms Time
More informationMean Square Convergent Finite Difference Scheme for Stochastic Parabolic PDEs
America Joural of Compuaioal Mahemaics, 04, 4, 80-88 Published Olie Sepember 04 i SciRes. hp://www.scirp.org/joural/ajcm hp://dx.doi.org/0.436/ajcm.04.4404 Mea Square Coverge Fiie Differece Scheme for
More informationSupplement for Stochastic Convex Optimization: Faster Local Growth Implies Faster Global Convergence
Supplemen for Sochasic Convex Opimizaion: Faser Local Growh Implies Faser Global Convergence Yi Xu Qihang Lin ianbao Yang Proof of heorem heorem Suppose Assumpion holds and F (w) obeys he LGC (6) Given
More informationInference of the Second Order Autoregressive. Model with Unit Roots
Ieraioal Mahemaical Forum Vol. 6 0 o. 5 595-604 Iferece of he Secod Order Auoregressive Model wih Ui Roos Ahmed H. Youssef Professor of Applied Saisics ad Ecoomerics Isiue of Saisical Sudies ad Research
More informationA Note on Random k-sat for Moderately Growing k
A Noe o Radom k-sat for Moderaely Growig k Ju Liu LMIB ad School of Mahemaics ad Sysems Sciece, Beihag Uiversiy, Beijig, 100191, P.R. Chia juliu@smss.buaa.edu.c Zogsheg Gao LMIB ad School of Mahemaics
More informationLINEAR APPROXIMATION OF THE BASELINE RBC MODEL JANUARY 29, 2013
LINEAR APPROXIMATION OF THE BASELINE RBC MODEL JANUARY 29, 203 Iroducio LINEARIZATION OF THE RBC MODEL For f( x, y, z ) = 0, mulivariable Taylor liear expasio aroud f( x, y, z) f( x, y, z) + f ( x, y,
More informationExistence Of Solutions For Nonlinear Fractional Differential Equation With Integral Boundary Conditions
Reserch Ivey: Ieriol Jourl Of Egieerig Ad Sciece Vol., Issue (April 3), Pp 8- Iss(e): 78-47, Iss(p):39-6483, Www.Reserchivey.Com Exisece Of Soluios For Nolier Frciol Differeil Equio Wih Iegrl Boudry Codiios,
More informationHigh-Probability Regret Bounds for Bandit Online Linear Optimization
High-Probabiliy Regre Bouds for Badi Olie Liear Opimizaio Peer L. Barle UC Berkeley barle@cs.berkeley.edu Varsha Dai Uiversiy of Chicago varsha@cs.uchicago.edu Aleader Rakhli UC Berkeley rakhli@cs.berkeley.edu
More informationA TAUBERIAN THEOREM FOR THE WEIGHTED MEAN METHOD OF SUMMABILITY
U.P.B. Sci. Bull., Series A, Vol. 78, Iss. 2, 206 ISSN 223-7027 A TAUBERIAN THEOREM FOR THE WEIGHTED MEAN METHOD OF SUMMABILITY İbrahim Çaak I his paper we obai a Tauberia codiio i erms of he weighed classical
More informationThe Solution of the One Species Lotka-Volterra Equation Using Variational Iteration Method ABSTRACT INTRODUCTION
Malaysia Joural of Mahemaical Scieces 2(2): 55-6 (28) The Soluio of he Oe Species Loka-Volerra Equaio Usig Variaioal Ieraio Mehod B. Baiha, M.S.M. Noorai, I. Hashim School of Mahemaical Scieces, Uiversii
More informationBoundary-to-Displacement Asymptotic Gains for Wave Systems With Kelvin-Voigt Damping
Boudary-o-Displaceme Asympoic Gais for Wave Sysems Wih Kelvi-Voig Dampig Iasso Karafyllis *, Maria Kooriaki ** ad Miroslav Krsic *** * Dep. of Mahemaics, Naioal Techical Uiversiy of Ahes, Zografou Campus,
More informationSampling Example. ( ) δ ( f 1) (1/2)cos(12πt), T 0 = 1
Samplig Example Le x = cos( 4π)cos( π). The fudameal frequecy of cos 4π fudameal frequecy of cos π is Hz. The ( f ) = ( / ) δ ( f 7) + δ ( f + 7) / δ ( f ) + δ ( f + ). ( f ) = ( / 4) δ ( f 8) + δ ( f
More informationFour equations describe the dynamic solution to RBC model. Consumption-leisure efficiency condition. Consumption-investment efficiency condition
LINEARIZING AND APPROXIMATING THE RBC MODEL SEPTEMBER 7, 200 For f( x, y, z ), mulivariable Taylor liear expasio aroud ( x, yz, ) f ( x, y, z) f( x, y, z) + f ( x, y, z)( x x) + f ( x, y, z)( y y) + f
More informationAN EXTENSION OF LUCAS THEOREM. Hong Hu and Zhi-Wei Sun. (Communicated by David E. Rohrlich)
Proc. Amer. Mah. Soc. 19(001, o. 1, 3471 3478. AN EXTENSION OF LUCAS THEOREM Hog Hu ad Zhi-Wei Su (Commuicaed by David E. Rohrlich Absrac. Le p be a prime. A famous heorem of Lucas saes ha p+s p+ ( s (mod
More informationIdeal Amplifier/Attenuator. Memoryless. where k is some real constant. Integrator. System with memory
Liear Time-Ivaria Sysems (LTI Sysems) Oulie Basic Sysem Properies Memoryless ad sysems wih memory (saic or dyamic) Causal ad o-causal sysems (Causaliy) Liear ad o-liear sysems (Lieariy) Sable ad o-sable
More informationLINEAR APPROXIMATION OF THE BASELINE RBC MODEL SEPTEMBER 17, 2013
LINEAR APPROXIMATION OF THE BASELINE RBC MODEL SEPTEMBER 7, 203 Iroducio LINEARIZATION OF THE RBC MODEL For f( xyz,, ) = 0, mulivariable Taylor liear expasio aroud f( xyz,, ) f( xyz,, ) + f( xyz,, )( x
More informationSolutions to Problems 3, Level 4
Soluios o Problems 3, Level 4 23 Improve he resul of Quesio 3 whe l. i Use log log o prove ha for real >, log ( {}log + 2 d log+ P ( + P ( d 2. Here P ( is defied i Quesio, ad parial iegraio has bee used.
More informationK3 p K2 p Kp 0 p 2 p 3 p
Mah 80-00 Mo Ar 0 Chaer 9 Fourier Series ad alicaios o differeial equaios (ad arial differeial equaios) 9.-9. Fourier series defiiio ad covergece. The idea of Fourier series is relaed o he liear algebra
More informationOn The Eneström-Kakeya Theorem
Applied Mahemaics,, 3, 555-56 doi:436/am673 Published Olie December (hp://wwwscirporg/oural/am) O The Eesröm-Kakeya Theorem Absrac Gulsha Sigh, Wali Mohammad Shah Bharahiar Uiversiy, Coimbaore, Idia Deparme
More informationOptimal Packet Scheduling in a Multiple Access Channel with Rechargeable Nodes
Opimal Packe Schedulig i a Muliple Access Chael wih Rechargeable Nodes Jig Yag Seur Ulukus Deparme of Elecrical ad Compuer Egieerig Uiversiy of Marylad, College Park, MD 20742 yagjig@umd.edu ulukus@umd.edu
More informationOutline. simplest HMM (1) simple HMMs? simplest HMM (2) Parameter estimation for discrete hidden Markov models
Oulie Parameer esimaio for discree idde Markov models Juko Murakami () ad Tomas Taylor (2). Vicoria Uiversiy of Welligo 2. Arizoa Sae Uiversiy Descripio of simple idde Markov models Maximum likeliood esimae
More information10.3 Autocorrelation Function of Ergodic RP 10.4 Power Spectral Density of Ergodic RP 10.5 Normal RP (Gaussian RP)
ENGG450 Probabiliy ad Saisics for Egieers Iroducio 3 Probabiliy 4 Probabiliy disribuios 5 Probabiliy Desiies Orgaizaio ad descripio of daa 6 Samplig disribuios 7 Ifereces cocerig a mea 8 Comparig wo reames
More informationDavid Randall. ( )e ikx. k = u x,t. u( x,t)e ikx dx L. x L /2. Recall that the proof of (1) and (2) involves use of the orthogonality condition.
! Revised April 21, 2010 1:27 P! 1 Fourier Series David Radall Assume ha u( x,) is real ad iegrable If he domai is periodic, wih period L, we ca express u( x,) exacly by a Fourier series expasio: ( ) =
More informationSolution. 1 Solutions of Homework 6. Sangchul Lee. April 28, Problem 1.1 [Dur10, Exercise ]
Soluio Sagchul Lee April 28, 28 Soluios of Homework 6 Problem. [Dur, Exercise 2.3.2] Le A be a sequece of idepede eves wih PA < for all. Show ha P A = implies PA i.o. =. Proof. Noice ha = P A c = P A c
More informationHomotopy Analysis Method for Solving Fractional Sturm-Liouville Problems
Ausralia Joural of Basic ad Applied Scieces, 4(1): 518-57, 1 ISSN 1991-8178 Homoopy Aalysis Mehod for Solvig Fracioal Surm-Liouville Problems 1 A Neamay, R Darzi, A Dabbaghia 1 Deparme of Mahemaics, Uiversiy
More informationNumerical approximation of Backward Stochastic Differential Equations with Jumps
Numerical approximaio of Bacward Sochasic Differeial Equaios wih Jumps Aoie Lejay, Ereso Mordeci, Soledad Torres To cie his versio: Aoie Lejay, Ereso Mordeci, Soledad Torres. Numerical approximaio of Bacward
More informationEXISTENCE THEORY OF RANDOM DIFFERENTIAL EQUATIONS D. S. Palimkar
Ieraioal Joural of Scieific ad Research Publicaios, Volue 2, Issue 7, July 22 ISSN 225-353 EXISTENCE THEORY OF RANDOM DIFFERENTIAL EQUATIONS D S Palikar Depare of Maheaics, Vasarao Naik College, Naded
More information1. Solve by the method of undetermined coefficients and by the method of variation of parameters. (4)
7 Differeial equaios Review Solve by he mehod of udeermied coefficies ad by he mehod of variaio of parameers (4) y y = si Soluio; we firs solve he homogeeous equaio (4) y y = 4 The correspodig characerisic
More informationOnline Geometric Optimization in the Bandit Setting Against an Adaptive Adversary
lie Geomeric pimizaio i he Badi Seig Agais a Adapive Adversary H. Breda McMaha ad Avrim Blum Caregie Mello Uiversiy, Pisburgh, PA, 15213, mcmaha,avrim @cs.cmu.edu Absrac. We give a algorihm for he badi
More informationAvailable online at J. Math. Comput. Sci. 4 (2014), No. 4, ISSN:
Available olie a hp://sci.org J. Mah. Compu. Sci. 4 (2014), No. 4, 716-727 ISSN: 1927-5307 ON ITERATIVE TECHNIQUES FOR NUMERICAL SOLUTIONS OF LINEAR AND NONLINEAR DIFFERENTIAL EQUATIONS S.O. EDEKI *, A.A.
More informationN! AND THE GAMMA FUNCTION
N! AND THE GAMMA FUNCTION Cosider he produc of he firs posiive iegers- 3 4 5 6 (-) =! Oe calls his produc he facorial ad has ha produc of he firs five iegers equals 5!=0. Direcly relaed o he discree! fucio
More informationBasic Results in Functional Analysis
Preared by: F.. ewis Udaed: Suday, Augus 7, 4 Basic Resuls i Fucioal Aalysis f ( ): X Y is coiuous o X if X, (, ) z f( z) f( ) f ( ): X Y is uiformly coiuous o X if i is coiuous ad ( ) does o deed o. f
More informationS n. = n. Sum of first n terms of an A. P is
PROGREION I his secio we discuss hree impora series amely ) Arihmeic Progressio (A.P), ) Geomeric Progressio (G.P), ad 3) Harmoic Progressio (H.P) Which are very widely used i biological scieces ad humaiies.
More informationOLS bias for econometric models with errors-in-variables. The Lucas-critique Supplementary note to Lecture 17
OLS bias for ecoomeric models wih errors-i-variables. The Lucas-criique Supplemeary oe o Lecure 7 RNy May 6, 03 Properies of OLS i RE models I Lecure 7 we discussed he followig example of a raioal expecaios
More informationarxiv:math/ v1 [math.pr] 5 Jul 2006
he Aals of Applied Probabiliy 2006, Vol. 16, No. 2, 984 1033 DOI: 10.1214/105051606000000088 c Isiue of Mahemaical Saisics, 2006 arxiv:mah/0607123v1 [mah.pr] 5 Jul 2006 ERROR ESIMAES FOR INOMIAL APPROXIMAIONS
More informationSome Newton s Type Inequalities for Geometrically Relative Convex Functions ABSTRACT. 1. Introduction
Malaysia Joural of Mahemaical Scieces 9(): 49-5 (5) MALAYSIAN JOURNAL OF MATHEMATICAL SCIENCES Joural homepage: hp://eispem.upm.edu.my/joural Some Newo s Type Ieualiies for Geomerically Relaive Covex Fucios
More informationarxiv: v1 [cs.lg] 9 Oct 2018
Bridgig he gap bewee regre miimizaio ad bes arm ideificaio, wih applicaio o A/B ess Rémy Degee Thomas Nedelec Cléme Calauzèes Viaey erche LSM, Uiversié aris Didero, Crieo AI Lab Crieo AI Lab CMLA, ENS
More informationResearch Article A MOLP Method for Solving Fully Fuzzy Linear Programming with LR Fuzzy Parameters
Mahemaical Problems i Egieerig Aricle ID 782376 10 pages hp://dx.doi.org/10.1155/2014/782376 Research Aricle A MOLP Mehod for Solvig Fully Fuzzy Liear Programmig wih Fuzzy Parameers Xiao-Peg Yag 12 Xue-Gag
More informationThe Eigen Function of Linear Systems
1/25/211 The Eige Fucio of Liear Sysems.doc 1/7 The Eige Fucio of Liear Sysems Recall ha ha we ca express (expad) a ime-limied sigal wih a weighed summaio of basis fucios: v ( ) a ψ ( ) = where v ( ) =
More informationResearch Article Generalized Equilibrium Problem with Mixed Relaxed Monotonicity
e Scieific World Joural, Aricle ID 807324, 4 pages hp://dx.doi.org/10.1155/2014/807324 Research Aricle Geeralized Equilibrium Problem wih Mixed Relaxed Moooiciy Haider Abbas Rizvi, 1 Adem KJlJçma, 2 ad
More informationF D D D D F. smoothed value of the data including Y t the most recent data.
Module 2 Forecasig 1. Wha is forecasig? Forecasig is defied as esimaig he fuure value ha a parameer will ake. Mos scieific forecasig mehods forecas he fuure value usig pas daa. I Operaios Maageme forecasig
More informationBIBECHANA A Multidisciplinary Journal of Science, Technology and Mathematics
Biod Prasad Dhaal / BIBCHANA 9 (3 5-58 : BMHSS,.5 (Olie Publicaio: Nov., BIBCHANA A Mulidisciliary Joural of Sciece, Techology ad Mahemaics ISSN 9-76 (olie Joural homeage: h://ejol.ifo/idex.h/bibchana
More informationStreaming Robust PCA
Sreamig Robus PCA U.N.Niraja Yag Shi The Uiversiy of Califoria a Irvie {u.iraja, shiy4}@uci.edu July 3, 06 Absrac I his paper, we cosider he problem of robus PCA i he sreamig seig wih space cosrais. Specifically,
More informationth m m m m central moment : E[( X X) ] ( X X) ( x X) f ( x)
1 Trasform Techiques h m m m m mome : E[ ] x f ( x) dx h m m m m ceral mome : E[( ) ] ( ) ( x) f ( x) dx A coveie wa of fidig he momes of a radom variable is he mome geeraig fucio (MGF). Oher rasform echiques
More informationNotes 03 largely plagiarized by %khc
1 1 Discree-Time Covoluio Noes 03 largely plagiarized by %khc Le s begi our discussio of covoluio i discree-ime, sice life is somewha easier i ha domai. We sar wih a sigal x[] ha will be he ipu io our
More informationStochastic Optimization with Variance Reduction for Infinite Datasets with Finite-Sum Structure
Sochasic Opimizaio wih Variace Reducio for Ifiie Daases wih Fiie-Sum Srucure Albero Biei, Julie Mairal To cie his versio: Albero Biei, Julie Mairal. Sochasic Opimizaio wih Variace Reducio for Ifiie Daases
More informationλiv Av = 0 or ( λi Av ) = 0. In order for a vector v to be an eigenvector, it must be in the kernel of λi
Liear lgebra Lecure #9 Noes This week s lecure focuses o wha migh be called he srucural aalysis of liear rasformaios Wha are he irisic properies of a liear rasformaio? re here ay fixed direcios? The discussio
More informationODEs II, Supplement to Lectures 6 & 7: The Jordan Normal Form: Solving Autonomous, Homogeneous Linear Systems. April 2, 2003
ODEs II, Suppleme o Lecures 6 & 7: The Jorda Normal Form: Solvig Auoomous, Homogeeous Liear Sysems April 2, 23 I his oe, we describe he Jorda ormal form of a marix ad use i o solve a geeral homogeeous
More informationCommon Fixed Point Theorem in Intuitionistic Fuzzy Metric Space via Compatible Mappings of Type (K)
Ieraioal Joural of ahemaics Treds ad Techology (IJTT) Volume 35 umber 4- July 016 Commo Fixed Poi Theorem i Iuiioisic Fuzzy eric Sace via Comaible aigs of Tye (K) Dr. Ramaa Reddy Assisa Professor De. of
More informationA Change-of-Variable Formula with Local Time on Surfaces
Sém. de Probab. L, Lecure Noes i Mah. Vol. 899, Spriger, 7, (69-96) Research Repor No. 437, 4, Dep. Theore. Sais. Aarhus (3 pp) A Chage-of-Variable Formula wih Local Time o Surfaces GORAN PESKIR 3 Le =
More informationSamuel Sindayigaya 1, Nyongesa L. Kennedy 2, Adu A.M. Wasike 3
Ieraioal Joural of Saisics ad Aalysis. ISSN 48-9959 Volume 6, Number (6, pp. -8 Research Idia Publicaios hp://www.ripublicaio.com The Populaio Mea ad is Variace i he Presece of Geocide for a Simple Birh-Deah-
More informationDIFFERENTIAL EQUATIONS
DIFFERENTIAL EQUATIONS M.A. (Previous) Direcorae of Disace Educaio Maharshi Dayaad Uiversiy ROHTAK 4 Copyrigh 3, Maharshi Dayaad Uiversiy, ROHTAK All Righs Reserved. No par of his publicaio may be reproduced
More informationarxiv: v2 [math.pr] 10 Apr 2014
arxiv:1311.2725v2 [mah.pr 1 Apr 214 Srog Rae of Covergece for he Euler-Maruyama Approximaio of Sochasic Differeial Equaios wih Irregular Coefficies Hoag-Log Ngo, Dai Taguchi Absrac We cosider he Euler-Maruyama
More informationxp (X = x) = P (X = 1) = θ. Hence, the method of moments estimator of θ is
Exercise 7 / page 356 Noe ha X i are ii from Beroulli(θ where 0 θ a Meho of momes: Sice here is oly oe parameer o be esimae we ee oly oe equaio where we equae he rs sample mome wih he rs populaio mome,
More information11. Adaptive Control in the Presence of Bounded Disturbances Consider MIMO systems in the form,
Lecure 6. Adapive Corol i he Presece of Bouded Disurbaces Cosider MIMO sysems i he form, x Aref xbu x Bref ycmd (.) y Cref x operaig i he presece of a bouded ime-depede disurbace R. All he assumpios ad
More informationThe analysis of the method on the one variable function s limit Ke Wu
Ieraioal Coferece o Advaces i Mechaical Egieerig ad Idusrial Iformaics (AMEII 5) The aalysis of he mehod o he oe variable fucio s i Ke Wu Deparme of Mahemaics ad Saisics Zaozhuag Uiversiy Zaozhuag 776
More informationApproximately Quasi Inner Generalized Dynamics on Modules. { } t t R
Joural of Scieces, Islamic epublic of Ira 23(3): 245-25 (22) Uiversiy of Tehra, ISSN 6-4 hp://jscieces.u.ac.ir Approximaely Quasi Ier Geeralized Dyamics o Modules M. Mosadeq, M. Hassai, ad A. Nikam Deparme
More informationCSE 241 Algorithms and Data Structures 10/14/2015. Skip Lists
CSE 41 Algorihms ad Daa Srucures 10/14/015 Skip Liss This hadou gives he skip lis mehods ha we discussed i class. A skip lis is a ordered, doublyliked lis wih some exra poiers ha allow us o jump over muliple
More informationOn the Validity of the Pairs Bootstrap for Lasso Estimators
O he Validiy of he Pairs Boosrap for Lasso Esimaors Lorezo Campoovo Uiversiy of S.Galle Ocober 2014 Absrac We sudy he validiy of he pairs boosrap for Lasso esimaors i liear regressio models wih radom covariaes
More informationReview Exercises for Chapter 9
0_090R.qd //0 : PM Page 88 88 CHAPTER 9 Ifiie Series I Eercises ad, wrie a epressio for he h erm of he sequece..,., 5, 0,,,, 0,... 7,... I Eercises, mach he sequece wih is graph. [The graphs are labeled
More informationStability Analysis of Discrete-Time Piecewise-Linear Systems: A Generating Function Approach
Ieraioal Joural of Corol, Auomaio, ad Sysems (4) (5):5- DOI.7/s555-3-64- ISSN:598-6446 eissn:5-49 hp://www.spriger.com/555 Sabiliy Aalysis of Discree-Time Piecewise-Liear Sysems: A Geeraig Fucio Approach
More informationThe Moment Approximation of the First Passage Time for the Birth Death Diffusion Process with Immigraton to a Moving Linear Barrier
America Joural of Applied Mahemaics ad Saisics, 015, Vol. 3, No. 5, 184-189 Available olie a hp://pubs.sciepub.com/ajams/3/5/ Sciece ad Educaio Publishig DOI:10.1691/ajams-3-5- The Mome Approximaio of
More informationOnline Nonparametric Regression
Olie Noparameric Regressio Alexader Rakhli Uiversiy of Pesylvaia Karhik Sridhara Uiversiy of Pesylvaia February 0, 204 Absrac We esablish opimal raes for olie regressio for arbirary classes of regressio
More informationIntroduction to the Mathematics of Lévy Processes
Iroducio o he Mahemaics of Lévy Processes Kazuhisa Masuda Deparme of Ecoomics The Graduae Ceer, The Ciy Uiversiy of New York, 365 Fifh Aveue, New York, NY 10016-4309 Email: maxmasuda@maxmasudacom hp://wwwmaxmasudacom/
More informationINTEGER INTERVAL VALUE OF NEWTON DIVIDED DIFFERENCE AND FORWARD AND BACKWARD INTERPOLATION FORMULA
Volume 8 No. 8, 45-54 ISSN: 34-3395 (o-lie versio) url: hp://www.ijpam.eu ijpam.eu INTEGER INTERVAL VALUE OF NEWTON DIVIDED DIFFERENCE AND FORWARD AND BACKWARD INTERPOLATION FORMULA A.Arul dass M.Dhaapal
More information