UC Berkeley Department of Electrical Engineering and Computer Science Department of Statistics
|
|
- Emery Washington
- 5 years ago
- Views:
Transcription
1 UC Berkeley Department of Electrical Engineering an Computer Science Department of Statistics EECS 8B / STAT 4B Avance Topics in Statistical Learning Theory Solutions 3 Spring 9 Solution 3. For parti, we will use the fact that a convex loss function φ is classification-calibrate if an only if it is ifferentiable at an φ < ; for part ii we will use the fact that if φ is convex an classification-calibrate, then Ψθ φ H +θ ; for part iii, as shown in the class, the ual program has the form } max α R n λ φ λα i αt yy T K α i where φ s sup t R st φt is the conjugate ual function of φ. a φt log + exp t. i φ t e t + an φ t is classification-calibrate. et >, hence φ is convex. φ e t + < implies φ ii By efinition, H φ η inf α R η log + e α + ηlog + e α }. The infimum is achieve at α η log for η,. Substituting it back gives us η η H φ η η log η ηlog η. + θ Ψθ φ H θlog θ + + θlog + θ. iii st log + exp t is unboune if s > or s <. For s,, st log + exp t achieves the maximum at t log +s s, so we have φ s +slog+s s log s. For s or s, φ s. With the interpretation log, we have φ + slog + s s log s if s, s + otherwise Therefore, the ual program is max α R n λ such that α i λ λα i log λα i + λα i logλα i αt yy T K α i i,...,n, b φt exp t.
2 i φ t e t an φ t e t >, hence φ is convex. φ < implies φ is classification-calibrate. ii By efinition, H φ η inf α R ηe α + ηe α }. The infimum is achieve at α η log η η for η,. Substituting it back gives us H φ η η η. + θ Ψθ φ H θ. iii st e t is unboune if s >. For s <, st e t achieves the maximum at t log s, so we have φ s s s log s. For s, φ s. By convention log, we have φ s s log s if s s + otherwise Therefore, the ual program is max α R n α i α i logλα i αt yy T K α i such that α i i,...,n Solution 3. a P min i,...,n Xi X > t P min i,...,n Xi X t P t} X i X i,...,n np X X t np } X j X j t j,..., n P X X t n t n t t n t, where the union boun is use in the first inequality.
3 b The quantity of interest is P min i,...,n X i X /4. From part a, we know that P min i,...,n Xi X /4 P min i,...,n Xi X > /4 n n. To guarantee P min i,...,n X i X /4 /, we must make sure / n/, which is equivalent to n. Therefore, the number of samples must grow exponentially fast as the imension increases. c By Fubini s theorem, ρ,n P n / min i,...,n Xi X > t n t + t + n /. n t t t The lower bouns for ifferent values of an n is as follows: n Solution 3.3 In the follows, we assume P ǫ B ǫ x i,i,...,mǫ;s,ρ} is one of ǫ-packings with the largest carinality. a Let C ǫ B ǫ y j,j,...,nǫ;s,ρ} be an ǫ-covering with the smallest carinality. By efinition of ǫ-covering, we know that for each x i,i,...,mǫ;s,ρ}, there exists some y j,j,...,nǫ;s,ρ} such that x i B ǫ y j. If Mǫ;S,ρ > Nǫ;S,ρ, then by Pigeonhole principle, there must exist two istinct centers x i,x i for some i,i,...,mǫ;s,ρ} an y j for some j,...,nǫ;s,ρ} such that x i B ǫ y j an x i B ǫ y j. This is equivalent to y j B ǫ x i an y j B ǫ x i, which contraicts the fact that B ǫ x i B ǫ x i. Therefore, Mǫ;S,ρ Nǫ;S,ρ. 3
4 b We claim C ǫ B ǫ x i,i,...,mǫ;s,ρ} is a ǫ-covering of set S, which implies Nǫ;S,ρ Mǫ;S,ρ. If there exists an s S that cannot be covere by C ǫ, then ρs,x i > ǫ for all i...,mǫ;s,ρ. We can show that B ǫ s is isjoint from all ǫ-balls in P ǫ : suppose there exists i,...,mǫ;s,ρ} such that y B ǫ s B ǫ x i, then ρs,x i ρs,y + ρy,x i ǫ. Therefore P ǫ B ǫ s is also an ǫ-packing, which contraicts that P ǫ is the maximal ǫ-packing. Solution 3.4 In this problem, the efinition of packing number Mǫ;S, is the maximum number of points in S such that the l metric between each pair is at least ǫ it iffers from problem 3.3 by a factor of. a For each point x inclue in the packing, consier an l ball centere at x with raius ǫ/. We will call these l balls as packing balls. Because the istance between each pair in the packing is at least ǫ, the packing balls are mutually isjoint. Therefore the maximum volumes covere by the packing balls are Mǫ;S, cǫ/, where c is a scaling constant. Because each point in the packing must belong to S, all the packing balls are inee containe in an l ball with raius + ǫ/. Hence, Mǫ;S, cǫ/ c + ǫ/ Mǫ;S, ǫ + 4. ǫ b First we prove the concentration boun for chi-square istribute ranom variable. Assume Z χ, we notice that EesZ s / an apply the Chernoff boun, PZ + δ where in the secon inequality we set s e s+δ s / exp s + δ /log s} exp δ log + δ /} exp δ /6, exp δ /6. Next we write x i x j z i z j z i z j z i z j z i δ +δ. Similarly, we have PZ δ z i z j z i + z j z i z j z i z j z j. Notice that z i z j χ, z i χ, z j χ. Applying the concentration boun erive above, we get 4
5 zi z j P δ zi P + δ P z i δ P zi z j exp δ /6 δ P z i + δ P δ z i + δ exp δ /6 P δ z i P + δ z i P δ z i + δ exp δ /6 δ δ Therefore, with probability at least exp δ /6 + exp δ /6 5exp δ /6, x i x j fδ δ + δ. δ Now we can set appropriate δ to let fδ /4 to guarantee P x i x j /4 5exp δ /6. This boun inicates that the probability that the istance between two uniformly ranom points on the surface of unit l ball is less than /4 is exponentially small. For part ii, we use the following probabilistic argument: ranomly choose M points x,...,x M on the surface of unit l ball, the probability that they form a /4-packing is 4} P x i x j > 4} P x i x j i j i j MM P x x /4 > M 5exp δ /6. So for M /5 expδ /3, we can guarantee the probability above is strictly greater than, which implies there must exist a /4-packing of size M. Hence M/4;S, /5 expδ /3. 5
15.1 Upper bound via Sudakov minorization
ECE598: Information-theoretic methos in high-imensional statistics Spring 206 Lecture 5: Suakov, Maurey, an uality of metric entropy Lecturer: Yihong Wu Scribe: Aolin Xu, Mar 7, 206 [E. Mar 24] In this
More informationarxiv: v1 [math.co] 15 Sep 2015
Circular coloring of signe graphs Yingli Kang, Eckhar Steffen arxiv:1509.04488v1 [math.co] 15 Sep 015 Abstract Let k, ( k) be two positive integers. We generalize the well stuie notions of (k, )-colorings
More informationREAL ANALYSIS I HOMEWORK 5
REAL ANALYSIS I HOMEWORK 5 CİHAN BAHRAN The questions are from Stein an Shakarchi s text, Chapter 3. 1. Suppose ϕ is an integrable function on R with R ϕ(x)x = 1. Let K δ(x) = δ ϕ(x/δ), δ > 0. (a) Prove
More informationHomework 2 Solutions EM, Mixture Models, PCA, Dualitys
Homewor Solutions EM, Mixture Moels, PCA, Dualitys CMU 0-75: Machine Learning Fall 05 http://www.cs.cmu.eu/~bapoczos/classes/ml075_05fall/ OUT: Oct 5, 05 DUE: Oct 9, 05, 0:0 AM An EM algorithm for a Mixture
More informationRobust Forward Algorithms via PAC-Bayes and Laplace Distributions. ω Q. Pr (y(ω x) < 0) = Pr A k
A Proof of Lemma 2 B Proof of Lemma 3 Proof: Since the support of LL istributions is R, two such istributions are equivalent absolutely continuous with respect to each other an the ivergence is well-efine
More informationLecture 10: October 30, 2017
Information an Coing Theory Autumn 2017 Lecturer: Mahur Tulsiani Lecture 10: October 30, 2017 1 I-Projections an applications In this lecture, we will talk more about fining the istribution in a set Π
More informationLecture Introduction. 2 Examples of Measure Concentration. 3 The Johnson-Lindenstrauss Lemma. CS-621 Theory Gems November 28, 2012
CS-6 Theory Gems November 8, 0 Lecture Lecturer: Alesaner Mąry Scribes: Alhussein Fawzi, Dorina Thanou Introuction Toay, we will briefly iscuss an important technique in probability theory measure concentration
More informationAn Optimal Algorithm for Bandit and Zero-Order Convex Optimization with Two-Point Feedback
Journal of Machine Learning Research 8 07) - Submitte /6; Publishe 5/7 An Optimal Algorithm for Banit an Zero-Orer Convex Optimization with wo-point Feeback Oha Shamir Department of Computer Science an
More informationThe chromatic number of graph powers
Combinatorics, Probability an Computing (19XX) 00, 000 000. c 19XX Cambrige University Press Printe in the Unite Kingom The chromatic number of graph powers N O G A A L O N 1 an B O J A N M O H A R 1 Department
More informationLECTURE NOTES ON DVORETZKY S THEOREM
LECTURE NOTES ON DVORETZKY S THEOREM STEVEN HEILMAN Abstract. We present the first half of the paper [S]. In particular, the results below, unless otherwise state, shoul be attribute to G. Schechtman.
More informationLecture 10 February 23
EECS 281B / STAT 241B: Advanced Topics in Statistical LearningSpring 2009 Lecture 10 February 23 Lecturer: Martin Wainwright Scribe: Dave Golland Note: These lecture notes are still rough, and have only
More informationLarge Triangles in the d-dimensional Unit-Cube (Extended Abstract)
Large Triangles in the -Dimensional Unit-Cube Extene Abstract) Hanno Lefmann Fakultät für Informatik, TU Chemnitz, D-0907 Chemnitz, Germany lefmann@informatik.tu-chemnitz.e Abstract. We consier a variant
More informationLecture 2: Correlated Topic Model
Probabilistic Moels for Unsupervise Learning Spring 203 Lecture 2: Correlate Topic Moel Inference for Correlate Topic Moel Yuan Yuan First of all, let us make some claims about the parameters an variables
More informationLower bounds on Locality Sensitive Hashing
Lower bouns on Locality Sensitive Hashing Rajeev Motwani Assaf Naor Rina Panigrahy Abstract Given a metric space (X, X ), c 1, r > 0, an p, q [0, 1], a istribution over mappings H : X N is calle a (r,
More informationFunction Spaces. 1 Hilbert Spaces
Function Spaces A function space is a set of functions F that has some structure. Often a nonparametric regression function or classifier is chosen to lie in some function space, where the assume structure
More informationLeast-Squares Regression on Sparse Spaces
Least-Squares Regression on Sparse Spaces Yuri Grinberg, Mahi Milani Far, Joelle Pineau School of Computer Science McGill University Montreal, Canaa {ygrinb,mmilan1,jpineau}@cs.mcgill.ca 1 Introuction
More information26.1 Metropolis method
CS880: Approximations Algorithms Scribe: Dave Anrzejewski Lecturer: Shuchi Chawla Topic: Metropolis metho, volume estimation Date: 4/26/07 The previous lecture iscusse they some of the key concepts of
More informationAcute sets in Euclidean spaces
Acute sets in Eucliean spaces Viktor Harangi April, 011 Abstract A finite set H in R is calle an acute set if any angle etermine by three points of H is acute. We examine the maximal carinality α() of
More informationCollaborative Ranking for Local Preferences Supplement
Collaborative Raning for Local Preferences Supplement Ber apicioglu Davi S Rosenberg Robert E Schapire ony Jebara YP YP Princeton University Columbia University Problem Formulation Let U {,,m} be the set
More informationarxiv: v4 [cs.ds] 7 Mar 2014
Analysis of Agglomerative Clustering Marcel R. Ackermann Johannes Blömer Daniel Kuntze Christian Sohler arxiv:101.697v [cs.ds] 7 Mar 01 Abstract The iameter k-clustering problem is the problem of partitioning
More informationLower Bounds for k-distance Approximation
Lower Bouns for k-distance Approximation Quentin Mérigot March 21, 2013 Abstract Consier a set P of N ranom points on the unit sphere of imension 1, an the symmetrize set S = P ( P). The halving polyheron
More informationLower Bounds for the Smoothed Number of Pareto optimal Solutions
Lower Bouns for the Smoothe Number of Pareto optimal Solutions Tobias Brunsch an Heiko Röglin Department of Computer Science, University of Bonn, Germany brunsch@cs.uni-bonn.e, heiko@roeglin.org Abstract.
More informationAn extension of Alexandrov s theorem on second derivatives of convex functions
Avances in Mathematics 228 (211 2258 2267 www.elsevier.com/locate/aim An extension of Alexanrov s theorem on secon erivatives of convex functions Joseph H.G. Fu 1 Department of Mathematics, University
More informationPDE Notes, Lecture #11
PDE Notes, Lecture # from Professor Jalal Shatah s Lectures Febuary 9th, 2009 Sobolev Spaces Recall that for u L loc we can efine the weak erivative Du by Du, φ := udφ φ C0 If v L loc such that Du, φ =
More informationA note on asymptotic formulae for one-dimensional network flow problems Carlos F. Daganzo and Karen R. Smilowitz
A note on asymptotic formulae for one-imensional network flow problems Carlos F. Daganzo an Karen R. Smilowitz (to appear in Annals of Operations Research) Abstract This note evelops asymptotic formulae
More information1. Aufgabenblatt zur Vorlesung Probability Theory
24.10.17 1. Aufgabenblatt zur Vorlesung By (Ω, A, P ) we always enote the unerlying probability space, unless state otherwise. 1. Let r > 0, an efine f(x) = 1 [0, [ (x) exp( r x), x R. a) Show that p f
More information164 Final Solutions 1
164 Final Solutions 1 1. Question 1 True/False a) Let f : R R be a C 3 function such that fx) for all x R. Then the graient escent algorithm starte at the point will fin the global minimum of f. FALSE.
More informationIterated Point-Line Configurations Grow Doubly-Exponentially
Iterate Point-Line Configurations Grow Doubly-Exponentially Joshua Cooper an Mark Walters July 9, 008 Abstract Begin with a set of four points in the real plane in general position. A to this collection
More informationA new proof of the sharpness of the phase transition for Bernoulli percolation on Z d
A new proof of the sharpness of the phase transition for Bernoulli percolation on Z Hugo Duminil-Copin an Vincent Tassion October 8, 205 Abstract We provie a new proof of the sharpness of the phase transition
More informationConvergence of Random Walks
Chapter 16 Convergence of Ranom Walks This lecture examines the convergence of ranom walks to the Wiener process. This is very important both physically an statistically, an illustrates the utility of
More informationWUCHEN LI AND STANLEY OSHER
CONSTRAINED DYNAMICAL OPTIMAL TRANSPORT AND ITS LAGRANGIAN FORMULATION WUCHEN LI AND STANLEY OSHER Abstract. We propose ynamical optimal transport (OT) problems constraine in a parameterize probability
More informationOn non-antipodal binary completely regular codes
Discrete Mathematics 308 (2008) 3508 3525 www.elsevier.com/locate/isc On non-antipoal binary completely regular coes J. Borges a, J. Rifà a, V.A. Zinoviev b a Department of Information an Communications
More informationNOTES ON EULER-BOOLE SUMMATION (1) f (l 1) (n) f (l 1) (m) + ( 1)k 1 k! B k (y) f (k) (y) dy,
NOTES ON EULER-BOOLE SUMMATION JONATHAN M BORWEIN, NEIL J CALKIN, AND DANTE MANNA Abstract We stuy a connection between Euler-MacLaurin Summation an Boole Summation suggeste in an AMM note from 196, which
More informationFloating Body, Illumination Body, and Polytopal Approximation
Convex Geometric Analysis MSRI Publications Volume 34, 998 Floating Boy, Illumination Boy, an Polytopal Approximation CARSTEN SCHÜTT Abstract. Let K be a convex boy inr an K t its floating boies. There
More informationExtreme Values by Resnick
1 Extreme Values by Resnick 1 Preliminaries 1.1 Uniform Convergence We will evelop the iea of something calle continuous convergence which will be useful to us later on. Denition 1. Let X an Y be metric
More informationProblem set 3: Solutions Math 207A, Fall where a, b are positive constants, and determine their linearized stability.
Problem set 3: s Math 207A, Fall 2014 1. Fin the fixe points of the Hénon map x n+1 = a x 2 n +by n, y n+1 = x n, where a, b are positive constants, an etermine their linearize stability. The fixe points
More information3.6. Let s write out the sample space for this random experiment:
STAT 5 3 Let s write out the sample space for this ranom experiment: S = {(, 2), (, 3), (, 4), (, 5), (2, 3), (2, 4), (2, 5), (3, 4), (3, 5), (4, 5)} This sample space assumes the orering of the balls
More informationTractability results for weighted Banach spaces of smooth functions
Tractability results for weighte Banach spaces of smooth functions Markus Weimar Mathematisches Institut, Universität Jena Ernst-Abbe-Platz 2, 07740 Jena, Germany email: markus.weimar@uni-jena.e March
More informationClassification Methods with Reject Option Based on Convex Risk Minimization
Journal of Machine Learning Research 11 (010) 111-130 Submitte /09; Revise 11/09; Publishe 1/10 Classification Methos with Reject Option Base on Convex Risk Minimization Ming Yuan H. Milton Stewart School
More informationMonotonicity for excited random walk in high dimensions
Monotonicity for excite ranom walk in high imensions Remco van er Hofsta Mark Holmes March, 2009 Abstract We prove that the rift θ, β) for excite ranom walk in imension is monotone in the excitement parameter
More informationFinal Exam Study Guide and Practice Problems Solutions
Final Exam Stuy Guie an Practice Problems Solutions Note: These problems are just some of the types of problems that might appear on the exam. However, to fully prepare for the exam, in aition to making
More informationLecture 6: September 19
36-755: Advanced Statistical Theory I Fall 2016 Lecture 6: September 19 Lecturer: Alessandro Rinaldo Scribe: YJ Choe Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have
More informationLecture 3 January 28
EECS 28B / STAT 24B: Advanced Topics in Statistical LearningSpring 2009 Lecture 3 January 28 Lecturer: Pradeep Ravikumar Scribe: Timothy J. Wheeler Note: These lecture notes are still rough, and have only
More information7.1 Support Vector Machine
67577 Intro. to Machine Learning Fall semester, 006/7 Lecture 7: Support Vector Machines an Kernel Functions II Lecturer: Amnon Shashua Scribe: Amnon Shashua 7. Support Vector Machine We return now to
More informationTime-of-Arrival Estimation in Non-Line-Of-Sight Environments
2 Conference on Information Sciences an Systems, The Johns Hopkins University, March 2, 2 Time-of-Arrival Estimation in Non-Line-Of-Sight Environments Sinan Gezici, Hisashi Kobayashi an H. Vincent Poor
More informationOn the Cauchy Problem for Von Neumann-Landau Wave Equation
Journal of Applie Mathematics an Physics 4 4-3 Publishe Online December 4 in SciRes http://wwwscirporg/journal/jamp http://xoiorg/436/jamp4343 On the Cauchy Problem for Von Neumann-anau Wave Equation Chuangye
More informationQF101: Quantitative Finance September 5, Week 3: Derivatives. Facilitator: Christopher Ting AY 2017/2018. f ( x + ) f(x) f(x) = lim
QF101: Quantitative Finance September 5, 2017 Week 3: Derivatives Facilitator: Christopher Ting AY 2017/2018 I recoil with ismay an horror at this lamentable plague of functions which o not have erivatives.
More informationLecture 7: Interchange of integration and limit
Lecture 7: Interchange of integration an limit Differentiating uner an integral sign To stuy the properties of a chf, we nee some technical result. When can we switch the ifferentiation an integration?
More informationWitten s Proof of Morse Inequalities
Witten s Proof of Morse Inequalities by Igor Prokhorenkov Let M be a smooth, compact, oriente manifol with imension n. A Morse function is a smooth function f : M R such that all of its critical points
More informationFurther Differentiation and Applications
Avance Higher Notes (Unit ) Prerequisites: Inverse function property; prouct, quotient an chain rules; inflexion points. Maths Applications: Concavity; ifferentiability. Real-Worl Applications: Particle
More informationLecture 2: August 31
0-704: Information Processing and Learning Fall 206 Lecturer: Aarti Singh Lecture 2: August 3 Note: These notes are based on scribed notes from Spring5 offering of this course. LaTeX template courtesy
More information6 General properties of an autonomous system of two first order ODE
6 General properties of an autonomous system of two first orer ODE Here we embark on stuying the autonomous system of two first orer ifferential equations of the form ẋ 1 = f 1 (, x 2 ), ẋ 2 = f 2 (, x
More informationTopic 7: Convergence of Random Variables
Topic 7: Convergence of Ranom Variables Course 003, 2016 Page 0 The Inference Problem So far, our starting point has been a given probability space (S, F, P). We now look at how to generate information
More informationarxiv: v2 [math.pr] 27 Nov 2018
Range an spee of rotor wals on trees arxiv:15.57v [math.pr] 7 Nov 1 Wilfrie Huss an Ecaterina Sava-Huss November, 1 Abstract We prove a law of large numbers for the range of rotor wals with ranom initial
More informationLearning Methods for Online Prediction Problems. Peter Bartlett Statistics and EECS UC Berkeley
Learning Methods for Online Prediction Problems Peter Bartlett Statistics and EECS UC Berkeley Course Synopsis A finite comparison class: A = {1,..., m}. 1. Prediction with expert advice. 2. With perfect
More information1 Lecture 20: Implicit differentiation
Lecture 20: Implicit ifferentiation. Outline The technique of implicit ifferentiation Tangent lines to a circle Derivatives of inverse functions by implicit ifferentiation Examples.2 Implicit ifferentiation
More informationOn the enumeration of partitions with summands in arithmetic progression
AUSTRALASIAN JOURNAL OF COMBINATORICS Volume 8 (003), Pages 149 159 On the enumeration of partitions with summans in arithmetic progression M. A. Nyblom C. Evans Department of Mathematics an Statistics
More informationRange and speed of rotor walks on trees
Range an spee of rotor wals on trees Wilfrie Huss an Ecaterina Sava-Huss May 15, 1 Abstract We prove a law of large numbers for the range of rotor wals with ranom initial configuration on regular trees
More informationOn combinatorial approaches to compressed sensing
On combinatorial approaches to compresse sensing Abolreza Abolhosseini Moghaam an Hayer Raha Department of Electrical an Computer Engineering, Michigan State University, East Lansing, MI, U.S. Emails:{abolhos,raha}@msu.eu
More informationCHAPTER 1 : DIFFERENTIABLE MANIFOLDS. 1.1 The definition of a differentiable manifold
CHAPTER 1 : DIFFERENTIABLE MANIFOLDS 1.1 The efinition of a ifferentiable manifol Let M be a topological space. This means that we have a family Ω of open sets efine on M. These satisfy (1), M Ω (2) the
More informationError Floors in LDPC Codes: Fast Simulation, Bounds and Hardware Emulation
Error Floors in LDPC Coes: Fast Simulation, Bouns an Harware Emulation Pamela Lee, Lara Dolecek, Zhengya Zhang, Venkat Anantharam, Borivoje Nikolic, an Martin J. Wainwright EECS Department University of
More information10-704: Information Processing and Learning Fall Lecture 21: Nov 14. sup
0-704: Information Processing and Learning Fall 206 Lecturer: Aarti Singh Lecture 2: Nov 4 Note: hese notes are based on scribed notes from Spring5 offering of this course LaeX template courtesy of UC
More informationLower Bounds for Local Monotonicity Reconstruction from Transitive-Closure Spanners
Lower Bouns for Local Monotonicity Reconstruction from Transitive-Closure Spanners Arnab Bhattacharyya Elena Grigorescu Mahav Jha Kyomin Jung Sofya Raskhonikova Davi P. Wooruff Abstract Given a irecte
More informationProblem set 2: Solutions Math 207B, Winter 2016
Problem set : Solutions Math 07B, Winter 016 1. A particle of mass m with position x(t) at time t has potential energy V ( x) an kinetic energy T = 1 m x t. The action of the particle over times t t 1
More informationA PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks
A PAC-Bayesian Approach to Spectrally-Normalize Margin Bouns for Neural Networks Behnam Neyshabur, Srinah Bhojanapalli, Davi McAllester, Nathan Srebro Toyota Technological Institute at Chicago {bneyshabur,
More informationTransformations of Random Variables
Transformations of Ranom Variables September, 2009 We begin with a ranom variable an we want to start looking at the ranom variable Y = g() = g where the function g : R R. The inverse image of a set A,
More informationProof of SPNs as Mixture of Trees
A Proof of SPNs as Mixture of Trees Theorem 1. If T is an inuce SPN from a complete an ecomposable SPN S, then T is a tree that is complete an ecomposable. Proof. Argue by contraiction that T is not a
More informationJointly continuous distributions and the multivariate Normal
Jointly continuous istributions an the multivariate Normal Márton alázs an álint Tóth October 3, 04 This little write-up is part of important founations of probability that were left out of the unit Probability
More informationSurvey Sampling. 1 Design-based Inference. Kosuke Imai Department of Politics, Princeton University. February 19, 2013
Survey Sampling Kosuke Imai Department of Politics, Princeton University February 19, 2013 Survey sampling is one of the most commonly use ata collection methos for social scientists. We begin by escribing
More informationMulti-agent Systems Reaching Optimal Consensus with Time-varying Communication Graphs
Preprints of the 8th IFAC Worl Congress Multi-agent Systems Reaching Optimal Consensus with Time-varying Communication Graphs Guoong Shi ACCESS Linnaeus Centre, School of Electrical Engineering, Royal
More informationSelf-normalized Martingale Tail Inequality
Online-to-Confience-Set Conversions an Application to Sparse Stochastic Banits A Self-normalize Martingale Tail Inequality The self-normalize martingale tail inequality that we present here is the scalar-value
More informationConsistency and asymptotic normality
Consistency an asymtotic normality Class notes for Econ 842 Robert e Jong March 2006 1 Stochastic convergence The asymtotic theory of minimization estimators relies on various theorems from mathematical
More informationRelative Loss Bounds for Multidimensional Regression Problems
Relative Loss Bounds for Multidimensional Regression Problems Jyrki Kivinen and Manfred Warmuth Presented by: Arindam Banerjee A Single Neuron For a training example (x, y), x R d, y [0, 1] learning solves
More informationNode Density and Delay in Large-Scale Wireless Networks with Unreliable Links
Noe Density an Delay in Large-Scale Wireless Networks with Unreliable Links Shizhen Zhao, Xinbing Wang Department of Electronic Engineering Shanghai Jiao Tong University, China Email: {shizhenzhao,xwang}@sjtu.eu.cn
More informationA Course in Machine Learning
A Course in Machine Learning Hal Daumé III 12 EFFICIENT LEARNING So far, our focus has been on moels of learning an basic algorithms for those moels. We have not place much emphasis on how to learn quickly.
More informationFall 2016: Calculus I Final
Answer the questions in the spaces provie on the question sheets. If you run out of room for an answer, continue on the back of the page. NO calculators or other electronic evices, books or notes are allowe
More informationIntroduction to Machine Learning
How o you estimate p(y x)? Outline Contents Introuction to Machine Learning Logistic Regression Varun Chanola April 9, 207 Generative vs. Discriminative Classifiers 2 Logistic Regression 2 3 Logistic Regression
More informationAnalysis IV, Assignment 4
Analysis IV, Assignment 4 Prof. John Toth Winter 23 Exercise Let f C () an perioic with f(x+2) f(x). Let a n f(t)e int t an (S N f)(x) N n N then f(x ) lim (S Nf)(x ). N a n e inx. If f is continuously
More informationExam 2 Review Solutions
Exam Review Solutions 1. True or False, an explain: (a) There exists a function f with continuous secon partial erivatives such that f x (x, y) = x + y f y = x y False. If the function has continuous secon
More informationHomework 7: Solutions. P3.1 from Lehmann, Romano, Testing Statistical Hypotheses.
Stat 300A Theory of Statistics Homework 7: Solutions Nikos Ignatiadis Due on November 28, 208 Solutions should be complete and concisely written. Please, use a separate sheet or set of sheets for each
More informationINDEPENDENT COMPONENT ANALYSIS VIA
INDEPENDENT COMPONENT ANALYSIS VIA NONPARAMETRIC MAXIMUM LIKELIHOOD ESTIMATION Truth Rotate S 2 2 1 0 1 2 3 4 X 2 2 1 0 1 2 3 4 4 2 0 2 4 6 4 2 0 2 4 6 S 1 X 1 Reconstructe S^ 2 2 1 0 1 2 3 4 Marginal
More informationStatics. There are four fundamental quantities which occur in mechanics:
Statics Mechanics isabranchofphysicsinwhichwestuythestate of rest or motion of boies subject to the action of forces. It can be ivie into two logical parts: statics, where we investigate the equilibrium
More informationStep 1. Analytic Properties of the Riemann zeta function [2 lectures]
Step. Analytic Properties of the Riemann zeta function [2 lectures] The Riemann zeta function is the infinite sum of terms /, n. For each n, the / is a continuous function of s, i.e. lim s s 0 n = s n,
More informationMath 242: Principles of Analysis Fall 2016 Homework 7 Part B Solutions
Mat 22: Principles of Analysis Fall 206 Homework 7 Part B Solutions. Sow tat f(x) = x 2 is not uniformly continuous on R. Solution. Te equation is equivalent to f(x) = 0 were f(x) = x 2 sin(x) 3. Since
More informationBayesian Estimation of the Entropy of the Multivariate Gaussian
Bayesian Estimation of the Entropy of the Multivariate Gaussian Santosh Srivastava Fre Hutchinson Cancer Research Center Seattle, WA 989, USA Email: ssrivast@fhcrc.org Maya R. Gupta Department of Electrical
More informationLecture 5: Importance sampling and Hamilton-Jacobi equations
Lecture 5: Importance sampling and Hamilton-Jacobi equations Henrik Hult Department of Mathematics KTH Royal Institute of Technology Sweden Summer School on Monte Carlo Methods and Rare Events Brown University,
More informationOn colour-blind distinguishing colour pallets in regular graphs
J Comb Optim (2014 28:348 357 DOI 10.1007/s10878-012-9556-x On colour-blin istinguishing colour pallets in regular graphs Jakub Przybyło Publishe online: 25 October 2012 The Author(s 2012. This article
More informationMonte Carlo Methods with Reduced Error
Monte Carlo Methos with Reuce Error As has been shown, the probable error in Monte Carlo algorithms when no information about the smoothness of the function is use is Dξ r N = c N. It is important for
More informationLecture XII. where Φ is called the potential function. Let us introduce spherical coordinates defined through the relations
Lecture XII Abstract We introuce the Laplace equation in spherical coorinates an apply the metho of separation of variables to solve it. This will generate three linear orinary secon orer ifferential equations:
More informationMAINTAINING LIMITED-RANGE CONNECTIVITY AMONG SECOND-ORDER AGENTS
MAINTAINING LIMITED-RANGE CONNECTIVITY AMONG SECOND-ORDER AGENTS KETAN SAVLA, GIUSEPPE NOTARSTEFANO AND FRANCESCO BULLO Abstract. In this paper we consier a-hoc networks of robotic agents with ouble integrator
More informationLecture 17: Density Estimation Lecturer: Yihong Wu Scribe: Jiaqi Mu, Mar 31, 2016 [Ed. Apr 1]
ECE598: Information-theoretic methods in high-dimensional statistics Spring 06 Lecture 7: Density Estimation Lecturer: Yihong Wu Scribe: Jiaqi Mu, Mar 3, 06 [Ed. Apr ] In last lecture, we studied the minimax
More informationThe Sokhotski-Plemelj Formula
hysics 24 Winter 207 The Sokhotski-lemelj Formula. The Sokhotski-lemelj formula The Sokhotski-lemelj formula is a relation between the following generalize functions (also calle istributions), ±iǫ = iπ(),
More informationSection 7.2. The Calculus of Complex Functions
Section 7.2 The Calculus of Complex Functions In this section we will iscuss limits, continuity, ifferentiation, Taylor series in the context of functions which take on complex values. Moreover, we will
More informationu!i = a T u = 0. Then S satisfies
Deterministic Conitions for Subspace Ientifiability from Incomplete Sampling Daniel L Pimentel-Alarcón, Nigel Boston, Robert D Nowak University of Wisconsin-Maison Abstract Consier an r-imensional subspace
More informationHITTING TIMES FOR RANDOM WALKS WITH RESTARTS
HITTING TIMES FOR RANDOM WALKS WITH RESTARTS SVANTE JANSON AND YUVAL PERES Abstract. The time it takes a ranom walker in a lattice to reach the origin from another vertex x, has infinite mean. If the walker
More informationLECTURE 15: COMPLETENESS AND CONVEXITY
LECTURE 15: COMPLETENESS AND CONVEXITY 1. The Hopf-Rinow Theorem Recall that a Riemannian manifold (M, g) is called geodesically complete if the maximal defining interval of any geodesic is R. On the other
More informationIn the usual geometric derivation of Bragg s Law one assumes that crystalline
Diffraction Principles In the usual geometric erivation of ragg s Law one assumes that crystalline arrays of atoms iffract X-rays just as the regularly etche lines of a grating iffract light. While this
More information. ISSN (print), (online) International Journal of Nonlinear Science Vol.6(2008) No.3,pp
. ISSN 1749-3889 (print), 1749-3897 (online) International Journal of Nonlinear Science Vol.6(8) No.3,pp.195-1 A Bouneness Criterion for Fourth Orer Nonlinear Orinary Differential Equations with Delay
More informationSPECIALIST MATHEMATICS
Victorian Certificate of Eucation 07 SUPERVISOR TO ATTACH PROCESSING LABEL HERE Letter STUDENT NUMBER SPECIALIST MATHEMATICS Written examination Friay 0 November 07 Reaing time: 9.00 am to 9.5 am (5 minutes)
More informationA stability result for mean width of L p -centroid bodies.
A stability result for mean with of L p -centroi boies. B. Fleury, O. Guéon an G. Paouris Abstract We give a ifferent proof of a recent result of lartag [1] concerning the concentration of the volume of
More information