Supplementary Material
|
|
- Bernard Potter
- 6 years ago
- Views:
Transcription
1 Suppleetary Material Wezhuo Ya Departet of Mechaical Eieeri, Natioal Uiversity of Siapore, Siapore Hua Xu Departet of Mechaical Eieeri, Natioal Uiversity of Siapore, Siapore Notatio Table [] The set {1,, } A subset of [] c The copleet of, c = [] \ I The idetity atrix X The saple atrix X R X i The ith colu of X β Vector β R β i The ith eleet of β β The vector whose ith eleet is β i if i or 0 otherwise (i) The ith disturbace atrix (i) j The jth colu of (i) The atrix whose ith colu is i if i or 0 otherwise W i Matrix W i R vec( ) The operator vectorizi a atrix by stacki its colus X p The l p -or of vec(x), vec(x) p 2. Proofs i Sectio 2 To prove the corollaries i Sectio 2, we ive the followi lea. Lea 1. If ay two differet roups p ad q i G i i the ucertaity set U (4) are o-overlappi for i = 1,, t, which eas p q =, the the optiizatio proble (5) is equivalet to i β R { y Xβ p + Proof. Sice ay two differet roups p ad q i G i are o-overlappi, we have G i, α (i) Hece the lea holds. ax α (i) Wi β = p c By usi Theore 3 ad Lea 1, we have G i α (i) ax G i c (W i β) p} (1) p c α (i) (W i β) = G i c (W i β) p (2)
2 Subissio ad Foratti Istructios for ICML Proof of Corollary 1: G 1 = {[]} satisfies the coditio of Lea 1, so we have c (W i β) p = c β 2 = c β 2. (3) G i G 1 2. Proof of Corollary 2: G 1 = {{1},, {}} satisfies the coditio of Lea 1, the c (W i β) p = c β p = c i β i. (4) G i G 1 3. Proof of Corollary 3: G 1 = { 1,, k } satisfies the coditio of Lea 1, so we have k c (W i β) p = c i β i p. (5) G i 4. Proof of Theore 2: G i = { i, c i } satisfies the coditio of Lea 1 ad c c i = 0, so that k c (W i β) p = (c i β i p + c c i β c i p) = G i k c i β i p. (6) 5. Proof of Corollary 4: The dual proble of the optiizatio proble ca be forulated as i vi =β, supp(v i ) i k c i v i p k ax i { c i v i p α α i,supp(v i ) i = ax α {α β + i i,supp(v i ) i { = ax α {α β ax i,supp(v i ) i { = ax i, α i c i α β k v i + α β} k c i v i p α i v i }} k α i v i c i v i p}} (7) Sice the costraits i the prial proble satisfy Slater s coditio, the stro duality holds. Fro the duality ad the coditio i Corollary 4, we have i β R { y Xβ p + G i, α (i) ax α (i) Wi β} p c = i β R { y Xβ p + ax α β} G 1, α p c = i β R { y Xβ p + i vi =β, supp(v i ) i k c i v i p}. (8)
3 Subissio ad Foratti Istructios for ICML Proof of Corollary 5: Fro Theore 2 ad Lea 1, we have c (W i β) p G i = c β p + c (W 2 β) p G 1 G 2 1 = c i β i + c i β i β i+1. (9) 7. Proof of Corollary 6: By usi the proofs of Corollary 1 ad Corollary 3, we ca obtai Corollary Proof of Corollary 7: G 1 = {{1},, {}} satisfies the coditio of Lea 1. Sice t = 1, c {i} = λ ad W 1 = D, we have 3. Proofs i Sectio Proof of Theore 4: Fro the defiitio of Û, we have G i c (W i β) p = G 1 λ (Dβ) p = λ (Dβ) i = λ Dβ 1. (10) ax y (X + )β p Û = ax c Z ax i, G i, (i) = y Xβ p + ax c Z = y Xβ p + = y Xβ p + = y Xβ p + y (X + )β p p c ax α (i) Wi β p c ax G i, α (i) c c 0;f i(c) 0 G i, α (i) i ax { λ R q +,κ Rk c R + k i λ R q +,κ Rk + υ(λ, κ, β) ax α (i) Wi β p c G i, α (i) ax α (i) Wi β + κ c p c q λ i f i (c)} (11) Hece we establish the theore by taki iiu over β o both sides. Now we show the optiizatio proble is covex ad tractable. we first prove that υ(λ, κ, β) is a covex fuctio of λ, κ, β. Sice υ(λ, κ, β) = ax { c R k, i, G i, α (i) p c α (i) W i β + κ c q λ i f i (c)} = ax c R k, i, G i, α (i) p c µ(λ, κ, β). (12) For fixed c ad α (i), µ(λ, κ, β) is a liear fuctio of λ, κ, β. Thus υ(λ, κ, β) is covex, which iplies the optiizatio proble is covex. By choosi paraeter γ, the optiizatio proble ca be reforulated as i s.t. y Xβ p υ(λ, κ, β) γ λ R p +, κ R k +, β R To show the proble is tractable, it suffices to costruct a polyoial-tie separatio oracle for the feasible set S (Grötschel et al. (Grötschel et al., 1988)). A separatio oracle is a routie such that for a solutio (λ 0, κ 0, β 0 ),
4 Subissio ad Foratti Istructios for ICML 2013 it ca fid, i polyoial tie, that (a) whether (λ 0, κ 0, β 0 ) belos to S or ot; ad (b) if (λ 0, κ 0, β 0 ) S, a hyperplae that separates (λ 0, κ 0, β 0 ) with S. To verify the feasibility of (λ 0, κ 0, β 0 ), otice that (λ 0, κ 0, β 0 ) S if ad oly if the optial value of the optiizatio proble (12) is saller tha or equal to γ, which ca be verified i polyoial tie. If (λ 0, κ 0, β 0 ) S, the by solvi (12), we ca fid i polyoial tie c 0, α (i) 0 such that α (i) 0 Wi β + κ c 0 which is the hyperplae separates (λ 0, κ 0, β 0 ) with S Extesio of Corollary 8: q λ i f i (c 0 ) > γ. Theore 1. Let 1,, t be t roups such that t i = [], ad i be a atrix whose colus except the ith oe are all zero. Suppose that c i is a i diesio vector whose eleets ive the or boud of j for j i, e.. j 2 c j i, ad c = (c 1,, c t ). We defie the ucertaity set as Û = { t j j i c such that c 0 ad c i q s i, i [t]; j 2 c j i, i [t], j i }, the the equivalet liear reularized reressio proble is where q is the dual or of q. i { y Xβ p + β R Proof. Fro Theore 3 ad Theore 4, we have i υ(λ, κ, β) λ R +,κ R + = i λ R +,κ R + ax c R s i β i q }, { j=1 λ i ( c i q + s i )}. i j (κ i + β i )c i Defie r i as the vector whose eleets are κ j + β j for j i, the the equatio above is equivalet to i λ R +,κ R + r i q λ i, ii[t] λ s = s i β i q, which establishes the theore. 4. Proofs i Sectio 5 Recall that the ucertaity set cosidered i this paper is U = { (1) W (t) W t i, G i, (i) 2 c } (13) where G i is the set of the roups of (i) ad c ives the boud of (i) for roup. We deote Ḡi ad Ḡc i as the set { G i c 0} ad G i Ḡi, respectively. I this theore, we restrict our discussio to the case that W i = I for i = 1,, t ad the boud c of (i) for each roup equals c or 0, so the ucertaity set ca be rewritte as U = { (1) + + (t) i, Ḡi, (i) 2 c } (14)
5 Subissio ad Foratti Istructios for ICML 2013 Note that the costrait 2 c ca be reforulated as the uio of several eleet-wise costraits. Deote D = {D i j D2 ij = c2, D ij 0} (we call a eleet D D decopositio), the we have { 2 c} = { i, j, ij D ij }. D D Siilarly, the ucertaity set { 2 c} is equivalet to { i, j, ij D ij }, D D where D = {D i j D2 ij = c2, D ij 0}. After the costraits of the ucertaity sets are decoposed ito eleet-wise costraits, the set {X + (1) + + (t) } ca also be represeted by a eleet-wise way. The otatio is a little coplicated so we first cosider three siple cases: Oe ucertaity set such that 2 c: for fixed D D, we have {X ij + ij } = [X ij D ij, X ij + D ij ]. Two ucertaity sets (1) ad (2) such that (1) 2 c ad (2) 2 c: for fixed D (1) D ad D (2) D, we have {X ij + (1) ij + (2) ij } = [X ij D (1) ij D (2) ij, X ij + D (1) ij + D (2) ij ]. Oe ucertaity set ad two overlappi roups p ad q such that p 2 c ad q 2 c: for fixed P D p ad Q D q, we have [X ij P ij, X ij + P ij ] j p, j q {X ij + ij } = [X ij Q ij, X ij + Q ij ] j p, j q [X ij i{p ij, Q ij }, X ij + i{p ij, Q ij }] j p, j q Thus, if the decopositio D D for each (i) is fixed, we have {X ij + (1) ij + + (t) ij } = [X ij γ ij, X ij +γ ij ] where γ ij is deteried by the decopositio Ds. Sice the uber of the eleets of (i) is less tha or equal to ( is the feature diesio ad is the uber of saples), there exists a decopositio D for each such that [X ij c, X ij + c ] [X ij γ ij, X ij + γ ij ]. We ow prove the theore. (i) Propositio 1. (Xu et al., 2010) Give a fuctio h : R +1 R ad Borel sets Z 1,, Z R +1, let P = {µ P S {1,, } : µ( i S Z i ) S /}. The followi holds 1 sup h(b i, r i ) = sup h(b i, r i )dµ(b i, r i ). (b i,r i ) Z i µ P R +1 Step 1: Usi the otatio above, we first ive the followi corollary: Corollary 1. Give y R, X R, the followi equatio holds for ay β R, y Xβ 2 + c + ax α (i) β = sup (b Ḡi, α (i) 2 r β) 2 dµ(b, r ) (15) c R +1 Here, ˆP () = S={D (i) } D (i) D, i, Ḡi µ ˆP () P (X, S, y, c ) = {µ P Z i = [y i c, y i + c ] where γ ij depeds o the decopositio set S. P (X, S, y, c ) S {1,, } : µ( i S Z i ) S /}, [X ij γ ij, X ij + γ ij ]; j=1
6 Subissio ad Foratti Istructios for ICML 2013 Proof. The riht had side of Equatio (15) is equal to sup S={D (i) } i, Ḡi,D (i) µ P (X,S,y,c ) D { sup Fro Theore 2, we kow that the left had side is equal to sup i, G i, δ y 2 c, (i) 2 c y + δ y (X + )β 2 = sup { sup i, G i,d (i) D = sup i, G i,d (i) D δ y 2 2 c2, (i) D (i) Furtherore, applyi Propositio 1 yields which proves the corollary. (b r β) 2 dµ(b, r )}. R +1 y + δ y (X + )β 2 } sup (b i,r i ) [y i c /,y i +c / ] j=1 [X ij γ ij,x ij +γ ij ] sup (b i,r i ) [y i c /,y i +c / ] j=1 [X ij γ ij,x ij +γ ij ] = sup (b r β) 2 dµ(b, r ) µ P (X,S,y,c ) R +1 = sup (b r β) 2 dµ(b, r ) µ P (X,S,y,c ) R +1 (b i r i β) (b i r i β). Step 2: As (Xu et al., 2010), we cosider the followi kerel estiator ive saples (b i, r i ), h (b, r) = (c +1 ) 1 K( b b i, r r i ) c where K(x) = I [ 1,1] +1(x)/2 +1, ad c = c. (16) Observe that the estiated distributio above belos to the set of distributios P (X, S, y, c ) = {µ P Z i = [y i c, y i + c ] S {1,, } : µ( i S Z i ) S /} [X ij γ ij, X ij + γ ij ]; j=1 ad hece belos to ˆP () = S={D (i) } D (i) D P (X, S, y, c )., i, Ḡi Step 3: Cobii the last two steps, ad usi the fact that h (b, r) h(b, r) d(b, r) oes to zero alost surely whe c 0 ad c +1 or equivaletly c 0 ad c +1. Now we prove cosistecy of robust reressio. Proof. Let f( ) be the true probability desity fuctio of the saples, ad ˆµ be the estiated distributio usi Equatio (16) ive S ad c, ad deote its desity fuctio as f ( ). The coditio that β(c, S ) 2 H alost surely ad P has a bouded support iplies that there exists a uiversal costat C such that ax (b r β(c, S )) 2 C
7 alost surely. By Corollary 1 ad ˆµ ˆP (), we have Subissio ad Foratti Istructios for ICML 2013 (b r β(c, S )) 2 dˆµ (b, r) sup (b r β(c, S )) 2 dµ (b, r) µ ˆP () = (b i r i β(c, S )) 2 + ax α (i) 1 β + c Ḡi, α(i) 2 c (b i r i β(p ))2 + ax α (i) 1 β + c Ḡi, α(i) 2 c Notice that, t ax Ḡi, α(i) 2 c α (i) β + 1 c coveres to 0 as c 0 alost surely, so the riht-had side coveres to (b r β(p )) 2 dp (b, r) as ad c 0 alost surely. Furtherore, we have (b r β(c, S )) 2 dp (b, r) (b r β(c, S )) 2 dˆµ (b, r) + ax (b r β(c, S )) 2 f (b, r) f(b, r) d(b, r) (b r β(c, S )) 2 dˆµ (b, r) + C f (b, r) f(b, r) d(b, r), where the last iequality follows fro the defiitio of C. Notice that f (b, r) f(b, r) d(b, r) oes to zero alost surely whe c 0 ad c +1. Hece the theore follows. As etioed i the paper, the assuptio that β(c, S ) 2 H i Theore 7 ca be reoved, the we have Theore 2. Let {c } covere to zero sufficietly slowly. The alost surely. li (b i r i β(c, S )) 2 dp (b, r) = (b i r i β(p ))2 dp (b, r) We ow prove this heore. We establish the followi lea first. Lea 2. Partitio the support of P as V 1,, V T such that the l radius of each set is less tha c. If a distributio µ satisfies µ(v t ) = #((b i, r i ) V t )/; t = 1,, T, (17) the µ ˆP (). Proof. Let Z i = [y i c, y i + c ] j=1 [X ij c, X ij + c ], recall that X ij is the jth eleet of r i. Notice that the l radius of V t is less tha c, we have (b i, r i ) V t V t Z i.
8 Subissio ad Foratti Istructios for ICML 2013 Therefore, for ay S {1,, }, the followi holds µ( i S Z i ) µ( V t i S : (b i, r i ) V t ) = µ(v t ) = #((b i, r i ) V t )/ S /. t i S:(b i,r i ) V t t i S:(b i,r i ) V t Hece µ P (X, S, y, c ) which iplies µ ˆP (). Partitio the support of P ito T subsets such that the l radius of each set is less tha c. Deote P () as the set of probability easures satisfyi Equatio (17). Hece P () ˆP () by Lea 1. Further otice that there exists a uiversal costat K such that β(c, S ) 2 K/c due to the fact that the square loss of the solutio β = 0 is bouded by a costat oly depeds o the support of P. Thus, there exists a costat C such that ax (b r β(c, S )) 2 C/c 2. Follow a siilar aruet as the proof of Theore 6, we have sup (b r β(c, S )) 2 dµ (b, r) µ P () (18) (b i r i β(p ))2 + ax α (i) 1 β + c Ḡi, α(i) 2 c ad (b r β(c, S )) 2 dp (b, r) if { (b r β(c, S )) 2 dµ (b, r) + ax µ P () r β(c, S )) 2 f µ (b, r) f(b, r) d(b, r)} sup (b r β(c, S )) 2 dµ (b, r) + 2C/c 2 if f µ (b, r) f(b, r) d(b, r), µ P () µ P () here f µ stads for the desity fuctio of a easure µ. Notice that P () is the set of distributios satisfyi Equatio (17), hece if µ P () f µ (b, r) f(b, r) d(b, r) is upper-bouded by T t=1 P (V t) #((b i, r i ) V t ) /, which oes to zero as icreases for ay fixed c. Therefore, 2C/c 2 if f µ (b, r) f(b, r) d(b, r) 0, µ P () if c 0 sufficietly slow. Cobii this with Iequality (18) proves the theore. Refereces Grötschel, Marti, Lovász, Lászlo, ad Schrijver, Alexader. Geoetric Aloriths ad Cobiatorial Optiizatio, volue 2. Sprier, Xu, H., Caraais, C., ad Maor, S. Robust reressio ad lasso. IEEE Trasactios o Iforatio Theory, 56(7): , 2010.
Lecture 10: Bounded Linear Operators and Orthogonality in Hilbert Spaces
Lecture : Bouded Liear Operators ad Orthogoality i Hilbert Spaces 34 Bouded Liear Operator Let ( X, ), ( Y, ) i i be ored liear vector spaces ad { } X Y The, T is said to be bouded if a real uber c such
More information1 The Primal and Dual of an Optimization Problem
CS 189 Itroductio to Machie Learig Fall 2017 Note 18 Previously, i our ivestigatio of SVMs, we forulated a costraied optiizatio proble that we ca solve to fid the optial paraeters for our hyperplae decisio
More informationLP in Standard and Slack Forms
LP i Stadard ad Slack Fors ax j=1 s.t. j=1 c j a ij b i for i=1, 2,..., 0 for j=1, 2,..., z = 0 j=1 c j x i = b i j=1 a ij for i=1, 2,..., Auxiliary Liear Progra L: LP i stadard for: ax j=1 L aux : Auxiliary
More informationThe Hypergeometric Coupon Collection Problem and its Dual
Joural of Idustrial ad Systes Egieerig Vol., o., pp -7 Sprig 7 The Hypergeoetric Coupo Collectio Proble ad its Dual Sheldo M. Ross Epstei Departet of Idustrial ad Systes Egieerig, Uiversity of Souther
More informationProblem. Consider the sequence a j for j N defined by the recurrence a j+1 = 2a j + j for j > 0
GENERATING FUNCTIONS Give a ifiite sequece a 0,a,a,, its ordiary geeratig fuctio is A : a Geeratig fuctios are ofte useful for fidig a closed forula for the eleets of a sequece, fidig a recurrece forula,
More informationSome remarks on the paper Some elementary inequalities of G. Bennett
Soe rears o the paper Soe eleetary iequalities of G. Beett Dag Ah Tua ad Luu Quag Bay Vieta Natioal Uiversity - Haoi Uiversity of Sciece Abstract We give soe couterexaples ad soe rears of soe of the corollaries
More informationf(1), and so, if f is continuous, f(x) = f(1)x.
2.2.35: Let f be a additive fuctio. i Clearly fx = fx ad therefore f x = fx for all Z+ ad x R. Hece, for ay, Z +, f = f, ad so, if f is cotiuous, fx = fx. ii Suppose that f is bouded o soe o-epty ope set.
More informationProbability Theory. Exercise Sheet 4. ETH Zurich HS 2017
ETH Zurich HS 2017 D-MATH, D-PHYS Prof. A.-S. Szita Coordiator Yili Wag Probability Theory Exercise Sheet 4 Exercise 4.1 Let X ) N be a sequece of i.i.d. rado variables i a probability space Ω, A, P ).
More informationA PROBABILITY PROBLEM
A PROBABILITY PROBLEM A big superarket chai has the followig policy: For every Euros you sped per buy, you ear oe poit (suppose, e.g., that = 3; i this case, if you sped 8.45 Euros, you get two poits,
More informationOptimal Estimator for a Sample Set with Response Error. Ed Stanek
Optial Estiator for a Saple Set wit Respose Error Ed Staek Itroductio We develop a optial estiator siilar to te FP estiator wit respose error tat was cosidered i c08ed63doc Te first 6 pages of tis docuet
More informationLecture 19. sup y 1,..., yn B d n
STAT 06A: Polyomials of adom Variables Lecture date: Nov Lecture 19 Grothedieck s Iequality Scribe: Be Hough The scribes are based o a guest lecture by ya O Doell. I this lecture we prove Grothedieck s
More informationStatistics and Data Analysis in MATLAB Kendrick Kay, February 28, Lecture 4: Model fitting
Statistics ad Data Aalysis i MATLAB Kedrick Kay, kedrick.kay@wustl.edu February 28, 2014 Lecture 4: Model fittig 1. The basics - Suppose that we have a set of data ad suppose that we have selected the
More informationECE 901 Lecture 4: Estimation of Lipschitz smooth functions
ECE 9 Lecture 4: Estiatio of Lipschitz sooth fuctios R. Nowak 5/7/29 Cosider the followig settig. Let Y f (X) + W, where X is a rado variable (r.v.) o X [, ], W is a r.v. o Y R, idepedet of X ad satisfyig
More informationLecture Outline. 2 Separating Hyperplanes. 3 Banach Mazur Distance An Algorithmist s Toolkit October 22, 2009
18.409 A Algorithist s Toolkit October, 009 Lecture 1 Lecturer: Joatha Keler Scribes: Alex Levi (009) 1 Outlie Today we ll go over soe of the details fro last class ad ake precise ay details that were
More informationLecture 19. Curve fitting I. 1 Introduction. 2 Fitting a constant to measured data
Lecture 9 Curve fittig I Itroductio Suppose we are preseted with eight poits of easured data (x i, y j ). As show i Fig. o the left, we could represet the uderlyig fuctio of which these data are saples
More informationDouble Derangement Permutations
Ope Joural of iscrete Matheatics, 206, 6, 99-04 Published Olie April 206 i SciRes http://wwwscirporg/joural/ojd http://dxdoiorg/04236/ojd2066200 ouble erageet Perutatios Pooya aeshad, Kayar Mirzavaziri
More informationAVERAGE MARKS SCALING
TERTIARY INSTITUTIONS SERVICE CENTRE Level 1, 100 Royal Street East Perth, Wester Australia 6004 Telephoe (08) 9318 8000 Facsiile (08) 95 7050 http://wwwtisceduau/ 1 Itroductio AVERAGE MARKS SCALING I
More informationLinear Support Vector Machines
Liear Support Vector Machies David S. Roseberg The Support Vector Machie For a liear support vector machie (SVM), we use the hypothesis space of affie fuctios F = { f(x) = w T x + b w R d, b R } ad evaluate
More informationdistinct distinct n k n k n! n n k k n 1 if k n, identical identical p j (k) p 0 if k > n n (k)
THE TWELVEFOLD WAY FOLLOWING GIAN-CARLO ROTA How ay ways ca we distribute objects to recipiets? Equivaletly, we wat to euerate equivalece classes of fuctios f : X Y where X = ad Y = The fuctios are subject
More informationAn Extended Result on the Optimal Estimation under Minimum Error Entropy Criterion
1 A Eteded Result o the Optial Estiatio uder Miiu Error Etropy Criterio Bado Che 1,Guai Wa 1, Nai Zhe 1, JoséC. Prícipe 2 1. Istitute of Artificial Itelliece ad Robotics, Xi a Jiaoto Uiversity, Xi a 7149,
More informationLecture 20 - Wave Propagation Response
.09/.093 Fiite Eleet Aalysis of Solids & Fluids I Fall 09 Lecture 0 - Wave Propagatio Respose Prof. K. J. Bathe MIT OpeCourseWare Quiz #: Closed book, 6 pages of otes, o calculators. Covers all aterials
More information10/ Statistical Machine Learning Homework #1 Solutions
Caregie Mello Uiversity Departet of Statistics & Data Sciece 0/36-70 Statistical Macie Learig Hoework # Solutios Proble [40 pts.] DUE: February, 08 Let X,..., X P were X i [0, ] ad P as desity p. Let p
More information174. A Tauberian Theorem for (J,,fin) Summability*)
No, 10] 807 174. A Tauberia Theore for (J,,fi) Suability*) By Kazuo IsHIGURo Departet of Matheatics, Hokkaido Uiversity, Sapporo (Co. by Kijiro KUNUGI, M.J,A., Dec. 12, 1964) 1. We suppose throughout ad
More information1 Duality revisited. AM 221: Advanced Optimization Spring 2016
AM 22: Advaced Optimizatio Sprig 206 Prof. Yaro Siger Sectio 7 Wedesday, Mar. 9th Duality revisited I this sectio, we will give a slightly differet perspective o duality. optimizatio program: f(x) x R
More informationRefinements of Jensen s Inequality for Convex Functions on the Co-Ordinates in a Rectangle from the Plane
Filoat 30:3 (206, 803 84 DOI 0.2298/FIL603803A Published by Faculty of Scieces ad Matheatics, Uiversity of Niš, Serbia Available at: http://www.pf.i.ac.rs/filoat Refieets of Jese s Iequality for Covex
More informationOn The Prime Numbers In Intervals
O The Prie Nubers I Itervals arxiv:1706.01009v1 [ath.nt] 4 Ju 2017 Kyle D. Balliet A Thesis Preseted to the Faculty of the Departet of Matheatics West Chester Uiversity West Chester, Pesylvaia I Partial
More informationMa/CS 6a Class 22: Power Series
Ma/CS 6a Class 22: Power Series By Ada Sheffer Power Series Mooial: ax i. Polyoial: a 0 + a 1 x + a 2 x 2 + + a x. Power series: A x = a 0 + a 1 x + a 2 x 2 + Also called foral power series, because we
More informationA New Type of q-szász-mirakjan Operators
Filoat 3:8 07, 567 568 https://doi.org/0.98/fil7867c Published by Faculty of Scieces ad Matheatics, Uiversity of Niš, Serbia Available at: http://www.pf.i.ac.rs/filoat A New Type of -Szász-Miraka Operators
More informationIntroduction to Optimization, DIKU Monday 19 November David Pisinger. Duality, motivation
Itroductio to Optiizatio, DIKU 007-08 Moday 9 Noveber David Pisiger Lecture, Duality ad sesitivity aalysis Duality, shadow prices, sesitivity aalysis, post-optial aalysis, copleetary slackess, KKT optiality
More informationNon-asymptotic sequential confidence regions with fixed sizes for the multivariate nonlinear parameters of regression. Andrey V.
No-asyptotic sequetial cofidece regios with fixed sizes for the ultivariate oliear paraeters of regressio Adrey V Tiofeev Abstract I this paper we cosider a sequetial desig for estiatio of o-liear paraeters
More informationBertrand s postulate Chapter 2
Bertrad s postulate Chapter We have see that the sequece of prie ubers, 3, 5, 7,... is ifiite. To see that the size of its gaps is ot bouded, let N := 3 5 p deote the product of all prie ubers that are
More informationWeek 10 Spring Lecture 19. Estimation of Large Covariance Matrices: Upper bound Observe. is contained in the following parameter space,
Week 0 Sprig 009 Lecture 9. stiatio of Large Covariace Matrices: Upper boud Observe ; ; : : : ; i.i.d. fro a p-variate Gaussia distributio, N (; pp ). We assue that the covariace atrix pp = ( ij ) i;jp
More informationFUZZY TRANSPORTATION PROBLEM WITH ADDITIONAL RESTRICTIONS
VOL. 5, NO. 2, FEBRUARY 200 ISSN 89-6608 ARPN Joural of Egieerig ad Applied Scieces 2006-200 Asia Research Publishig Network (ARPN). All rights reserved. www.arpjourals.co FUZZY TRANSPORTATION PROBLEM
More informationHomework Set #3 - Solutions
EE 15 - Applicatios of Covex Optimizatio i Sigal Processig ad Commuicatios Dr. Adre Tkaceko JPL Third Term 11-1 Homework Set #3 - Solutios 1. a) Note that x is closer to x tha to x l i the Euclidea orm
More information19.1 The dictionary problem
CS125 Lecture 19 Fall 2016 19.1 The dictioary proble Cosider the followig data structural proble, usually called the dictioary proble. We have a set of ites. Each ite is a (key, value pair. Keys are i
More informationJacobi symbols. p 1. Note: The Jacobi symbol does not necessarily distinguish between quadratic residues and nonresidues. That is, we could have ( a
Jacobi sybols efiitio Let be a odd positive iteger If 1, the Jacobi sybol : Z C is the costat fuctio 1 1 If > 1, it has a decopositio ( as ) a product of (ot ecessarily distict) pries p 1 p r The Jacobi
More informationarxiv: v1 [math.st] 12 Dec 2018
DIVERGENCE MEASURES ESTIMATION AND ITS ASYMPTOTIC NORMALITY THEORY : DISCRETE CASE arxiv:181.04795v1 [ath.st] 1 Dec 018 Abstract. 1) BA AMADOU DIADIÉ AND 1,,4) LO GANE SAMB 1. Itroductio 1.1. Motivatios.
More informationSOME FINITE SIMPLE GROUPS OF LIE TYPE C n ( q) ARE UNIQUELY DETERMINED BY THEIR ELEMENT ORDERS AND THEIR ORDER
Joural of Algebra, Nuber Theory: Advaces ad Applicatios Volue, Nuber, 010, Pages 57-69 SOME FINITE SIMPLE GROUPS OF LIE TYPE C ( q) ARE UNIQUELY DETERMINED BY THEIR ELEMENT ORDERS AND THEIR ORDER School
More informationDISTANCE BETWEEN UNCERTAIN RANDOM VARIABLES
MATHEMATICAL MODELLING OF ENGINEERING PROBLEMS Vol, No, 4, pp5- http://doiorg/88/ep4 DISTANCE BETWEEN UNCERTAIN RANDOM VARIABLES Yogchao Hou* ad Weicai Peg Departet of Matheatical Scieces, Chaohu Uiversity,
More informationOn Modeling On Minimum Description Length Modeling. M-closed
O Modelig O Miiu Descriptio Legth Modelig M M-closed M-ope Do you believe that the data geeratig echais really is i your odel class M? 7 73 Miiu Descriptio Legth Priciple o-m-closed predictive iferece
More informationMath 4707 Spring 2018 (Darij Grinberg): midterm 2 page 1. Math 4707 Spring 2018 (Darij Grinberg): midterm 2 with solutions [preliminary version]
Math 4707 Sprig 08 Darij Griberg: idter page Math 4707 Sprig 08 Darij Griberg: idter with solutios [preliiary versio] Cotets 0.. Coutig first-eve tuples......................... 3 0.. Coutig legal paths
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationUniversity of Manitoba, Mathletics 2009
Uiversity of Maitoba, Mathletics 009 Sessio 5: Iequalities Facts ad defiitios AM-GM iequality: For a, a,, a 0, a + a + + a (a a a ) /, with equality iff all a i s are equal Cauchy s iequality: For reals
More informationDomination Number of Square of Cartesian Products of Cycles
Ope Joural of Discrete Matheatics, 01,, 88-94 Published Olie October 01 i SciRes http://wwwscirporg/joural/ojd http://dxdoiorg/10436/ojd014008 Doiatio Nuber of Square of artesia Products of ycles Morteza
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationPointwise observation of the state given by parabolic system with boundary condition involving multiple time delays
1.1515/acsc-216-11 Archives of Cotrol Scieces Volue 26(LXII), 216 No. 2, pages 189 197 Poitwise observatio of the state give by parabolic syste with boudary coditio ivolvig ultiple tie delays ADAM KOWALEWSKI
More informationIP Reference guide for integer programming formulations.
IP Referece guide for iteger programmig formulatios. by James B. Orli for 15.053 ad 15.058 This documet is iteded as a compact (or relatively compact) guide to the formulatio of iteger programs. For more
More informationIt is always the case that unions, intersections, complements, and set differences are preserved by the inverse image of a function.
MATH 532 Measurable Fuctios Dr. Neal, WKU Throughout, let ( X, F, µ) be a measure space ad let (!, F, P ) deote the special case of a probability space. We shall ow begi to study real-valued fuctios defied
More informationECE Spring Prof. David R. Jackson ECE Dept. Notes 20
ECE 6341 Sprig 016 Prof. David R. Jackso ECE Dept. Notes 0 1 Spherical Wave Fuctios Cosider solvig ψ + k ψ = 0 i spherical coordiates z φ θ r y x Spherical Wave Fuctios (cot.) I spherical coordiates we
More information) is a square matrix with the property that for any m n matrix A, the product AI equals A. The identity matrix has a ii
square atrix is oe that has the sae uber of rows as colus; that is, a atrix. he idetity atrix (deoted by I, I, or [] I ) is a square atrix with the property that for ay atrix, the product I equals. he
More informationMa 530 Introduction to Power Series
Ma 530 Itroductio to Power Series Please ote that there is material o power series at Visual Calculus. Some of this material was used as part of the presetatio of the topics that follow. What is a Power
More information62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +
62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of
More informationTechnical Proofs for Homogeneity Pursuit
Techical Proofs for Homogeeity Pursuit bstract This is the supplemetal material for the article Homogeeity Pursuit, submitted for publicatio i Joural of the merica Statistical ssociatio. B Proofs B. Proof
More informationInverse Matrix. A meaning that matrix B is an inverse of matrix A.
Iverse Matrix Two square matrices A ad B of dimesios are called iverses to oe aother if the followig holds, AB BA I (11) The otio is dual but we ofte write 1 B A meaig that matrix B is a iverse of matrix
More informationSolution. 1 Solutions of Homework 1. Sangchul Lee. October 27, Problem 1.1
Solutio Sagchul Lee October 7, 017 1 Solutios of Homework 1 Problem 1.1 Let Ω,F,P) be a probability space. Show that if {A : N} F such that A := lim A exists, the PA) = lim PA ). Proof. Usig the cotiuity
More informationSupplementary Material for Fast Stochastic AUC Maximization with O(1/n)-Convergence Rate
Supplemetary Material for Fast Stochastic AUC Maximizatio with O/-Covergece Rate Migrui Liu Xiaoxua Zhag Zaiyi Che Xiaoyu Wag 3 iabao Yag echical Lemmas ized versio of Hoeffdig s iequality, ote that We
More informationOn twin primes associated with the Hawkins random sieve
Joural of Nuber Theory 9 006 84 96 wwwelsevierco/locate/jt O twi pries associated with the Hawkis rado sieve HM Bui,JPKeatig School of Matheatics, Uiversity of Bristol, Bristol, BS8 TW, UK Received 3 July
More informationAPPENDIX A SMO ALGORITHM
AENDIX A SMO ALGORITHM Sequetial Miimal Optimizatio SMO) is a simple algorithm that ca quickly solve the SVM Q problem without ay extra matrix storage ad without usig time-cosumig umerical Q optimizatio
More informationA Tabu Search Method for Finding Minimal Multi-Homogeneous Bézout Number
Joural of Matheatics ad Statistics 6 (): 105-109, 010 ISSN 1549-3644 010 Sciece Publicatios A Tabu Search Method for Fidig Miial Multi-Hoogeeous Bézout Nuber Hassa M.S. Bawazir ad Ali Abd Raha Departet
More informationACO Comprehensive Exam 9 October 2007 Student code A. 1. Graph Theory
1. Graph Theory Prove that there exist o simple plaar triagulatio T ad two distict adjacet vertices x, y V (T ) such that x ad y are the oly vertices of T of odd degree. Do ot use the Four-Color Theorem.
More informationLearning Theory for Conditional Risk Minimization: Supplementary Material
Learig Theory for Coditioal Risk Miiizatio: Suppleetary Material Alexader Zii IST Austria azii@istacat Christoph H Lapter IST Austria chl@istacat Proofs Proof of Theore After the applicatio of (6) ad (8)
More informationReview Problems 1. ICME and MS&E Refresher Course September 19, 2011 B = C = AB = A = A 2 = A 3... C 2 = C 3 = =
Review Problems ICME ad MS&E Refresher Course September 9, 0 Warm-up problems. For the followig matrices A = 0 B = C = AB = 0 fid all powers A,A 3,(which is A times A),... ad B,B 3,... ad C,C 3,... Solutio:
More informationx !1! + 1!2!
4 Euler-Maclauri Suatio Forula 4. Beroulli Nuber & Beroulli Polyoial 4.. Defiitio of Beroulli Nuber Beroulli ubers B (,,3,) are defied as coefficiets of the followig equatio. x e x - B x! 4.. Expreesio
More informationRiemann Sums y = f (x)
Riema Sums Recall that we have previously discussed the area problem I its simplest form we ca state it this way: The Area Problem Let f be a cotiuous, o-egative fuctio o the closed iterval [a, b] Fid
More informationTHE GREATEST ORDER OF THE DIVISOR FUNCTION WITH INCREASING DIMENSION
MATHEMATICA MONTISNIGRI Vol XXVIII (013) 17-5 THE GREATEST ORDER OF THE DIVISOR FUNCTION WITH INCREASING DIMENSION GLEB V. FEDOROV * * Mechaics ad Matheatics Faculty Moscow State Uiversity Moscow, Russia
More informationw (1) ˆx w (1) x (1) /ρ and w (2) ˆx w (2) x (2) /ρ.
2 5. Weighted umber of late jobs 5.1. Release dates ad due dates: maximimizig the weight of o-time jobs Oce we add release dates, miimizig the umber of late jobs becomes a sigificatly harder problem. For
More informationDefinitions and Theorems. where x are the decision variables. c, b, and a are constant coefficients.
Defiitios ad Theorems Remember the scalar form of the liear programmig problem, Miimize, Subject to, f(x) = c i x i a 1i x i = b 1 a mi x i = b m x i 0 i = 1,2,, where x are the decisio variables. c, b,
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationAn Algorithmist s Toolkit October 20, Lecture 11
18.409 A Algorithist s Toolkit October 20, 2009 Lecture 11 Lecturer: Joatha Keler Scribe: Chaithaya Badi 1 Outlie Today we ll itroduce ad discuss Polar of a covex body. Correspodece betwee or fuctios ad
More informationThe multiplicative structure of finite field and a construction of LRC
IERG6120 Codig for Distributed Storage Systems Lecture 8-06/10/2016 The multiplicative structure of fiite field ad a costructio of LRC Lecturer: Keeth Shum Scribe: Zhouyi Hu Notatios: We use the otatio
More informationDifferent kinds of Mathematical Induction
Differet ids of Mathematical Iductio () Mathematical Iductio Give A N, [ A (a A a A)] A N () (First) Priciple of Mathematical Iductio Let P() be a propositio (ope setece), if we put A { : N p() is true}
More information1 Approximating Integrals using Taylor Polynomials
Seughee Ye Ma 8: Week 7 Nov Week 7 Summary This week, we will lear how we ca approximate itegrals usig Taylor series ad umerical methods. Topics Page Approximatig Itegrals usig Taylor Polyomials. Defiitios................................................
More information42 Dependence and Bases
42 Depedece ad Bases The spa s(a) of a subset A i vector space V is a subspace of V. This spa ay be the whole vector space V (we say the A spas V). I this paragraph we study subsets A of V which spa V
More informationn=1 a n is the sequence (s n ) n 1 n=1 a n converges to s. We write a n = s, n=1 n=1 a n
Series. Defiitios ad first properties A series is a ifiite sum a + a + a +..., deoted i short by a. The sequece of partial sums of the series a is the sequece s ) defied by s = a k = a +... + a,. k= Defiitio
More informationIf a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero?
2 Lebesgue Measure I Chapter 1 we defied the cocept of a set of measure zero, ad we have observed that every coutable set is of measure zero. Here are some atural questios: If a subset E of R cotais a
More informationIntegrals of Functions of Several Variables
Itegrals of Fuctios of Several Variables We ofte resort to itegratios i order to deterie the exact value I of soe quatity which we are uable to evaluate by perforig a fiite uber of additio or ultiplicatio
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationA Note on the Symmetric Powers of the Standard Representation of S n
A Note o the Symmetric Powers of the Stadard Represetatio of S David Savitt 1 Departmet of Mathematics, Harvard Uiversity Cambridge, MA 0138, USA dsavitt@mathharvardedu Richard P Staley Departmet of Mathematics,
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationBinomial transform of products
Jauary 02 207 Bioial trasfor of products Khristo N Boyadzhiev Departet of Matheatics ad Statistics Ohio Norther Uiversity Ada OH 4580 USA -boyadzhiev@ouedu Abstract Give the bioial trasfors { b } ad {
More information3. Z Transform. Recall that the Fourier transform (FT) of a DT signal xn [ ] is ( ) [ ] = In order for the FT to exist in the finite magnitude sense,
3. Z Trasform Referece: Etire Chapter 3 of text. Recall that the Fourier trasform (FT) of a DT sigal x [ ] is ω ( ) [ ] X e = j jω k = xe I order for the FT to exist i the fiite magitude sese, S = x [
More informationOn Order of a Function of Several Complex Variables Analytic in the Unit Polydisc
ISSN 746-7659, Eglad, UK Joural of Iforatio ad Coutig Sciece Vol 6, No 3, 0, 95-06 O Order of a Fuctio of Several Colex Variables Aalytic i the Uit Polydisc Rata Kuar Dutta + Deartet of Matheatics, Siliguri
More informationOptimally Sparse SVMs
A. Proof of Lemma 3. We here prove a lower boud o the umber of support vectors to achieve geeralizatio bouds of the form which we cosider. Importatly, this result holds ot oly for liear classifiers, but
More informationMath 451: Euclidean and Non-Euclidean Geometry MWF 3pm, Gasson 204 Homework 3 Solutions
Math 451: Euclidea ad No-Euclidea Geometry MWF 3pm, Gasso 204 Homework 3 Solutios Exercises from 1.4 ad 1.5 of the otes: 4.3, 4.10, 4.12, 4.14, 4.15, 5.3, 5.4, 5.5 Exercise 4.3. Explai why Hp, q) = {x
More informationSlide Set 13 Linear Model with Endogenous Regressors and the GMM estimator
Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday
More informationf(x) dx as we do. 2x dx x also diverges. Solution: We compute 2x dx lim
Math 3, Sectio 2. (25 poits) Why we defie f(x) dx as we do. (a) Show that the improper itegral diverges. Hece the improper itegral x 2 + x 2 + b also diverges. Solutio: We compute x 2 + = lim b x 2 + =
More informationOn Nonsingularity of Saddle Point Matrices. with Vectors of Ones
Iteratioal Joural of Algebra, Vol. 2, 2008, o. 4, 197-204 O Nosigularity of Saddle Poit Matrices with Vectors of Oes Tadeusz Ostrowski Istitute of Maagemet The State Vocatioal Uiversity -400 Gorzów, Polad
More informationJORGE LUIS AROCHA AND BERNARDO LLANO. Average atchig polyoial Cosider a siple graph G =(V E): Let M E a atchig of the graph G: If M is a atchig, the a
MEAN VALUE FOR THE MATCHING AND DOMINATING POLYNOMIAL JORGE LUIS AROCHA AND BERNARDO LLANO Abstract. The ea value of the atchig polyoial is coputed i the faily of all labeled graphs with vertices. We dee
More informationLøsningsførslag i 4M
Norges tekisk aturviteskapelige uiversitet Istitutt for matematiske fag Side 1 av 6 Løsigsførslag i 4M Oppgave 1 a) A sketch of the graph of the give f o the iterval [ 3, 3) is as follows: The Fourier
More informationEstimation of KL Divergence Between Large-Alphabet Distributions
Estiatio of KL Divergece Betwee Large-Alphabet Distributios Yuheg Bu Shaofeg Zou Yigbi Liag Veugopal V. Veeravalli Uiversity of Illiois at Urbaa-Chapaig Syracuse Uiversity Eail: bu3@illiois.edu szou02@syr.edu
More informationBernoulli Polynomials Talks given at LSBU, October and November 2015 Tony Forbes
Beroulli Polyoials Tals give at LSBU, October ad Noveber 5 Toy Forbes Beroulli Polyoials The Beroulli polyoials B (x) are defied by B (x), Thus B (x) B (x) ad B (x) x, B (x) x x + 6, B (x) dx,. () B 3
More informationLargest Entries of Sample Correlation Matrices from Equi-correlated Normal Populations
Largest Etries of Saple Correlatio Matrices fro Equi-correlated Noral Populatios Jiaqig Fa ad Tiefeg Jiag Priceto Uiversity ad Uiversity of Miesota Abstract The paper studies the liitig distributio of
More informationChapter 2. Asymptotic Notation
Asyptotic Notatio 3 Chapter Asyptotic Notatio Goal : To siplify the aalysis of ruig tie by gettig rid of details which ay be affected by specific ipleetatio ad hardware. [1] The Big Oh (O-Notatio) : It
More informationINEQUALITIES BJORN POONEN
INEQUALITIES BJORN POONEN 1 The AM-GM iequality The most basic arithmetic mea-geometric mea (AM-GM) iequality states simply that if x ad y are oegative real umbers, the (x + y)/2 xy, with equality if ad
More information18.S34 (FALL, 2007) GREATEST INTEGER PROBLEMS. n + n + 1 = 4n + 2.
18.S34 (FALL, 007) GREATEST INTEGER PROBLEMS Note: We use the otatio x for the greatest iteger x, eve if the origial source used the older otatio [x]. 1. (48P) If is a positive iteger, prove that + + 1
More informationComplete Solutions to Supplementary Exercises on Infinite Series
Coplete Solutios to Suppleetary Eercises o Ifiite Series. (a) We eed to fid the su ito partial fractios gives By the cover up rule we have Therefore Let S S A / ad A B B. Covertig the suad / the by usig
More informationAvailable online at J. Math. Comput. Sci. 4 (2014), No. 3, ISSN:
Available olie at http://scik.org J. Math. Coput. Sci. (1, No. 3, 9-5 ISSN: 197-537 ON SYMMETRICAL FUNCTIONS WITH BOUNDED BOUNDARY ROTATION FUAD. S. M. AL SARARI 1,, S. LATHA 1 Departet of Studies i Matheatics,
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationHomework 2. Show that if h is a bounded sesquilinear form on the Hilbert spaces X and Y, then h has the representation
omework 2 1 Let X ad Y be ilbert spaces over C The a sesquiliear form h o X Y is a mappig h : X Y C such that for all x 1, x 2, x X, y 1, y 2, y Y ad all scalars α, β C we have (a) h(x 1 + x 2, y) h(x
More information