ALMOST SURE CONVERGENCE OF RANDOM GOSSIP ALGORITHMS

Size: px
Start display at page:

Download "ALMOST SURE CONVERGENCE OF RANDOM GOSSIP ALGORITHMS"

Transcription

1 ALMOST SURE CONVERGENCE OF RANDOM GOSSIP ALGORITHMS Giorgio Picci with T. Taylor, ASU Tempe AZ. Wofgang Runggaldier s Birthday, Brixen July

2 CONSENSUS FOR RANDOM GOSSIP ALGORITHMS Consider a finite set of nodes representing say wireless sensors or distributed computing units, can they achieve a common goal by exchanging information only locally? exchanging information locally for the purpose of forming a common estimate of some physical variable x; Each node k forms his own estimate x k (t), t Z + and updates according to exchange of information with a neighbor. Neighboring pairs are chosen randomly Q: will all local estimates {x k (t), k = 1,...,n} converge to the same value as t?. 2

3 DYNAMICS OF RANDOM GOSSIP ALGORITHMS While two nodes v i and v j are in communication, they exchange information to refine their own estimate using the neighbor s estimate. Model this adjustament in discrete time by a simple symmetric linear relation x i (t + 1) = x i (t) + p(x j (t) x i (t)) x j (t + 1) = x j (t) + p(x i (t) x j (t)) where p is some positive gain parameter modeling the speed of adjustment. For stability need to impose that 1 2p 1 and hence 0 p 1. For p = 1 2 you take the average of the two measurements so that x i(t +1) = x j (t + 1). 3

4 DYNAMICS OF RANDOM GOSSIP ALGORITHMS The whole coordinate vector x(t) R n evolves according to x(t +1) = A(e)x(t), the matrix A(e) R n n depending on the edge e = v i v j selected at that particular time instant; A(e) = p p p 1 p... ) T = I n p (1 vi 1 v j )(1 vi 1 v j 1 4

5 EIGENSPACES The vector 1 vi has the i th entry equal to 1 and zero otherwise. A(e) is a symmetric doubly stochastic matrix. The value 1 2p is a simple eigenvalue associated to the eigenvector (1 vi 1 v j ), A(e)(1 vi 1 v j ) = (1 vi 1 v j ) p(1 vi 1 v j )2 = (1 2p)(1 vi 1 v j ) ( ) the orthogonal (codimension one) subspace 1 vi 1 v j is the eigenspace of the eigenvalue 1. Let 1 := [1,...,1]. Want x(t) to converge to the subspace {1} := {α1; α R}. This would be automatically true for a fixed irreducible d-stochastic matrix. 5

6 A CONTROLLABILITY LEMMA Lemma 1 Let G = (V,E) be a graph. Then iff G is connected. span{1 vi 1 v j : (v i v j ) E} = 1 ( 1 vi 1 v j ) span {1}; i.e. Corollary 1 Let G = (V,E ) with E E be a subgraph of G. Let {e i : 1 i m } be an ordering of E, and let π denote a permutation of {1,2,,m }. Let B(E,π) = m i=1 A(e πi), where the product is ordered from right to left. Then if and only if G is connected. B(E,π) 1 < 1 6

7 THE EDGE PROCESS Let Ω = E N, be the space of all semi-infinite sequences taking values in E, and let σ : Ω Ω denote the shift map: σ(e 0,e 1,e 2,,e n, ) = (e 1,e 2,,e n, ). Let ev k : Ω E denote the evaluation on the k th term. Let µ denote an ergodic shift invariant probability measure on Ω, so that the edge process e(k) : ω ev k (ω) is ergodic. Special cases: e(k) is iid, or an ergodic Markov chain. However, what we shall do works for general ergodic processes. Consider the function C : Ω Z R n n, C(ω,t) := A(ev i (ω)) = A(ev 0 (σ i ω)) t 1 i=0 which by stationarity of e obeys the composition rule C(ω,t +s) =C(σ t ω,s)c(ω,t) with C(ω,0) = I. Such a function is called a matrix cocycle. t 1 i=0 7

8 MULTIPLICATIVE ERGODIC THEOREM Theorem 1 [Oseledet s Multiplicative Ergodic Theorem] Let µ be a shift invariant probability measure on Ω and suppose that the shift map σ : Ω Ω is ergodic and that log + C(ω,t) is in L 1. Then the limit Λ = lim t ( C(ω,t) T C(ω,t)) 1 2t (1) exists with probability one, is symmetric and nonnegative definite, and is µ a.s. independent of ω. Let λ 1 < λ 2 < λ k for k n be the distinct eigenvalues of Λ, let U i denote the eigenspace of λ i, and let V i = i j=1 U j. Then for u V i V i 1, 1 lim t t log C(ω,t)u = log(λ i). (2) The numbers λ i are called the Lyapunov exponents of C. 8

9 MULTIPLICATIVE ERGODIC THEOREM The Lyapunov exponents control the exponential rate of convergence (or non-convergence) to consensus. The matrices A(e) are doubly stochastic as are any matrix products of them, C(ω,t). If follows that the constant functions on V, {1}, as well as the mean zero functions in {1} are invariant under the action of this cocycle and of its transpose. Thus these subspaces are also invariant under the limiting matrix Λ of the Oseledet s theorem. There is a Lyapunov exponent associated with the subspace {1} which, it is not difficult to see, is one. There are n 1 Lyapunov exponents associated with the subspace 1, so the key point is to characterize them. 9

10 CONVERGENCE TO CONSENSUS For x R n use the symbol x := 1 n ni=1 x i. The main convergence result follows. Theorem 2 Let G = (V,E) be a connected graph and let e(t) be an ergodic stochastic process taking values on E. Suppose that the support of the probability distribution induced by e(t) is all of E. Let the gossip algorithm be initialized at x(0) = x 0. Then there is a (deterministic) constant λ < 1 and a (random) constant K λ such that µ-almost surely. x(t) x 0 1 < K λ λ t x 0 x

11 OPEN QUESTIONS Rate of convergence (for L 2...) Multiple gossiping : more than one pair of communicating edges per time slot, Convergence is merely associated to the time T it takes the algorithm to visit a spanning tree with positive probability. Indeed, the actual rate of convergence of the algorithm is just determined by T. Much remains to be done!!! 11

12 REFERENCES W. Runggaldier (circa 1970): STILLE WASSER GRUNDEN TIEF, unpublished (although well known among specialists). 12

Randomized Gossip Algorithms

Randomized Gossip Algorithms 2508 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 52, NO 6, JUNE 2006 Randomized Gossip Algorithms Stephen Boyd, Fellow, IEEE, Arpita Ghosh, Student Member, IEEE, Balaji Prabhakar, Member, IEEE, and Devavrat

More information

STOCHASTIC PROCESSES Basic notions

STOCHASTIC PROCESSES Basic notions J. Virtamo 38.3143 Queueing Theory / Stochastic processes 1 STOCHASTIC PROCESSES Basic notions Often the systems we consider evolve in time and we are interested in their dynamic behaviour, usually involving

More information

Markov Chains, Stochastic Processes, and Matrix Decompositions

Markov Chains, Stochastic Processes, and Matrix Decompositions Markov Chains, Stochastic Processes, and Matrix Decompositions 5 May 2014 Outline 1 Markov Chains Outline 1 Markov Chains 2 Introduction Perron-Frobenius Matrix Decompositions and Markov Chains Spectral

More information

Markov Chains, Random Walks on Graphs, and the Laplacian

Markov Chains, Random Walks on Graphs, and the Laplacian Markov Chains, Random Walks on Graphs, and the Laplacian CMPSCI 791BB: Advanced ML Sridhar Mahadevan Random Walks! There is significant interest in the problem of random walks! Markov chain analysis! Computer

More information

MET Workshop: Exercises

MET Workshop: Exercises MET Workshop: Exercises Alex Blumenthal and Anthony Quas May 7, 206 Notation. R d is endowed with the standard inner product (, ) and Euclidean norm. M d d (R) denotes the space of n n real matrices. When

More information

Distributed Optimization over Networks Gossip-Based Algorithms

Distributed Optimization over Networks Gossip-Based Algorithms Distributed Optimization over Networks Gossip-Based Algorithms Angelia Nedić angelia@illinois.edu ISE Department and Coordinated Science Laboratory University of Illinois at Urbana-Champaign Outline Random

More information

Consensus and Distributed Inference Rates Using Network Divergence

Consensus and Distributed Inference Rates Using Network Divergence DIMACS August 2017 1 / 26 Consensus and Distributed Inference Rates Using Network Divergence Anand D. Department of Electrical and Computer Engineering, The State University of New Jersey August 23, 2017

More information

Convex Optimization CMU-10725

Convex Optimization CMU-10725 Convex Optimization CMU-10725 Simulated Annealing Barnabás Póczos & Ryan Tibshirani Andrey Markov Markov Chains 2 Markov Chains Markov chain: Homogen Markov chain: 3 Markov Chains Assume that the state

More information

Definition A finite Markov chain is a memoryless homogeneous discrete stochastic process with a finite number of states.

Definition A finite Markov chain is a memoryless homogeneous discrete stochastic process with a finite number of states. Chapter 8 Finite Markov Chains A discrete system is characterized by a set V of states and transitions between the states. V is referred to as the state space. We think of the transitions as occurring

More information

18.440: Lecture 33 Markov Chains

18.440: Lecture 33 Markov Chains 18.440: Lecture 33 Markov Chains Scott Sheffield MIT 1 Outline Markov chains Examples Ergodicity and stationarity 2 Outline Markov chains Examples Ergodicity and stationarity 3 Markov chains Consider a

More information

18.600: Lecture 32 Markov Chains

18.600: Lecture 32 Markov Chains 18.600: Lecture 32 Markov Chains Scott Sheffield MIT Outline Markov chains Examples Ergodicity and stationarity Outline Markov chains Examples Ergodicity and stationarity Markov chains Consider a sequence

More information

T.8. Perron-Frobenius theory of positive matrices From: H.R. Thieme, Mathematics in Population Biology, Princeton University Press, Princeton 2003

T.8. Perron-Frobenius theory of positive matrices From: H.R. Thieme, Mathematics in Population Biology, Princeton University Press, Princeton 2003 T.8. Perron-Frobenius theory of positive matrices From: H.R. Thieme, Mathematics in Population Biology, Princeton University Press, Princeton 2003 A vector x R n is called positive, symbolically x > 0,

More information

Distributed Optimization over Random Networks

Distributed Optimization over Random Networks Distributed Optimization over Random Networks Ilan Lobel and Asu Ozdaglar Allerton Conference September 2008 Operations Research Center and Electrical Engineering & Computer Science Massachusetts Institute

More information

Distributed Randomized Algorithms for the PageRank Computation Hideaki Ishii, Member, IEEE, and Roberto Tempo, Fellow, IEEE

Distributed Randomized Algorithms for the PageRank Computation Hideaki Ishii, Member, IEEE, and Roberto Tempo, Fellow, IEEE IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 55, NO. 9, SEPTEMBER 2010 1987 Distributed Randomized Algorithms for the PageRank Computation Hideaki Ishii, Member, IEEE, and Roberto Tempo, Fellow, IEEE Abstract

More information

Average-Consensus of Multi-Agent Systems with Direct Topology Based on Event-Triggered Control

Average-Consensus of Multi-Agent Systems with Direct Topology Based on Event-Triggered Control Outline Background Preliminaries Consensus Numerical simulations Conclusions Average-Consensus of Multi-Agent Systems with Direct Topology Based on Event-Triggered Control Email: lzhx@nankai.edu.cn, chenzq@nankai.edu.cn

More information

Conceptual Questions for Review

Conceptual Questions for Review Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.

More information

Invertibility and stability. Irreducibly diagonally dominant. Invertibility and stability, stronger result. Reducible matrices

Invertibility and stability. Irreducibly diagonally dominant. Invertibility and stability, stronger result. Reducible matrices Geršgorin circles Lecture 8: Outline Chapter 6 + Appendix D: Location and perturbation of eigenvalues Some other results on perturbed eigenvalue problems Chapter 8: Nonnegative matrices Geršgorin s Thm:

More information

Asymmetric Randomized Gossip Algorithms for Consensus

Asymmetric Randomized Gossip Algorithms for Consensus Asymmetric Randomized Gossip Algorithms for Consensus F. Fagnani S. Zampieri Dipartimento di Matematica, Politecnico di Torino, C.so Duca degli Abruzzi, 24, 10129 Torino, Italy, (email: fabio.fagnani@polito.it).

More information

Social network analysis: social learning

Social network analysis: social learning Social network analysis: social learning Donglei Du (ddu@unb.edu) Faculty of Business Administration, University of New Brunswick, NB Canada Fredericton E3B 9Y2 October 20, 2016 Donglei Du (UNB) AlgoTrading

More information

Consensus and Products of Random Stochastic Matrices: Exact Rate for Convergence in Probability

Consensus and Products of Random Stochastic Matrices: Exact Rate for Convergence in Probability Consensus and Products of Random Stochastic Matrices: Exact Rate for Convergence in Probability 1 arxiv:1202.6389v1 [math.pr] 28 Feb 2012 Dragana Bajović, João Xavier, José M. F. Moura and Bruno Sinopoli

More information

Necessary and sufficient conditions for strong R-positivity

Necessary and sufficient conditions for strong R-positivity Necessary and sufficient conditions for strong R-positivity Wednesday, November 29th, 2017 The Perron-Frobenius theorem Let A = (A(x, y)) x,y S be a nonnegative matrix indexed by a countable set S. We

More information

Almost Sure Convergence to Consensus in Markovian Random Graphs

Almost Sure Convergence to Consensus in Markovian Random Graphs Proceedings of the 47th IEEE Conference on Decision and Control Cancun, Mexico, Dec. 9-11, 2008 Almost Sure Convergence to Consensus in Markovian Random Graphs Ion Matei, Nuno Martins and John S. Baras

More information

On the convergence of weighted-average consensus

On the convergence of weighted-average consensus On the convergence of weighted-average consensus Francisco Pedroche Miguel Rebollo Carlos Carrascosa Alberto Palomares arxiv:307.7562v [math.oc] 29 Jul 203 Abstract In this note we give sufficient conditions

More information

LECTURE 3. Last time:

LECTURE 3. Last time: LECTURE 3 Last time: Mutual Information. Convexity and concavity Jensen s inequality Information Inequality Data processing theorem Fano s Inequality Lecture outline Stochastic processes, Entropy rate

More information

Markov chains (week 6) Solutions

Markov chains (week 6) Solutions Markov chains (week 6) Solutions 1 Ranking of nodes in graphs. A Markov chain model. The stochastic process of agent visits A N is a Markov chain (MC). Explain. The stochastic process of agent visits A

More information

The goal of this chapter is to study linear systems of ordinary differential equations: dt,..., dx ) T

The goal of this chapter is to study linear systems of ordinary differential equations: dt,..., dx ) T 1 1 Linear Systems The goal of this chapter is to study linear systems of ordinary differential equations: ẋ = Ax, x(0) = x 0, (1) where x R n, A is an n n matrix and ẋ = dx ( dt = dx1 dt,..., dx ) T n.

More information

MARKOV PROCESSES. Valerio Di Valerio

MARKOV PROCESSES. Valerio Di Valerio MARKOV PROCESSES Valerio Di Valerio Stochastic Process Definition: a stochastic process is a collection of random variables {X(t)} indexed by time t T Each X(t) X is a random variable that satisfy some

More information

MATH 56A SPRING 2008 STOCHASTIC PROCESSES 65

MATH 56A SPRING 2008 STOCHASTIC PROCESSES 65 MATH 56A SPRING 2008 STOCHASTIC PROCESSES 65 2.2.5. proof of extinction lemma. The proof of Lemma 2.3 is just like the proof of the lemma I did on Wednesday. It goes like this. Suppose that â is the smallest

More information

Random Walks on Graphs. One Concrete Example of a random walk Motivation applications

Random Walks on Graphs. One Concrete Example of a random walk Motivation applications Random Walks on Graphs Outline One Concrete Example of a random walk Motivation applications shuffling cards universal traverse sequence self stabilizing token management scheme random sampling enumeration

More information

TMA Calculus 3. Lecture 21, April 3. Toke Meier Carlsen Norwegian University of Science and Technology Spring 2013

TMA Calculus 3. Lecture 21, April 3. Toke Meier Carlsen Norwegian University of Science and Technology Spring 2013 TMA4115 - Calculus 3 Lecture 21, April 3 Toke Meier Carlsen Norwegian University of Science and Technology Spring 2013 www.ntnu.no TMA4115 - Calculus 3, Lecture 21 Review of last week s lecture Last week

More information

Multi Stage Queuing Model in Level Dependent Quasi Birth Death Process

Multi Stage Queuing Model in Level Dependent Quasi Birth Death Process International Journal of Statistics and Systems ISSN 973-2675 Volume 12, Number 2 (217, pp. 293-31 Research India Publications http://www.ripublication.com Multi Stage Queuing Model in Level Dependent

More information

18.175: Lecture 30 Markov chains

18.175: Lecture 30 Markov chains 18.175: Lecture 30 Markov chains Scott Sheffield MIT Outline Review what you know about finite state Markov chains Finite state ergodicity and stationarity More general setup Outline Review what you know

More information

Markov chains. Randomness and Computation. Markov chains. Markov processes

Markov chains. Randomness and Computation. Markov chains. Markov processes Markov chains Randomness and Computation or, Randomized Algorithms Mary Cryan School of Informatics University of Edinburgh Definition (Definition 7) A discrete-time stochastic process on the state space

More information

NOTES ON THE PERRON-FROBENIUS THEORY OF NONNEGATIVE MATRICES

NOTES ON THE PERRON-FROBENIUS THEORY OF NONNEGATIVE MATRICES NOTES ON THE PERRON-FROBENIUS THEORY OF NONNEGATIVE MATRICES MIKE BOYLE. Introduction By a nonnegative matrix we mean a matrix whose entries are nonnegative real numbers. By positive matrix we mean a matrix

More information

Large Deviations Rates for Distributed Inference. Doctor of Philosophy. Electrical and Computer Engineering. Dragana Bajović

Large Deviations Rates for Distributed Inference. Doctor of Philosophy. Electrical and Computer Engineering. Dragana Bajović Large Deviations Rates for Distributed Inference Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Electrical and Computer Engineering Dragana Bajović Dipl.

More information

Learning distributions and hypothesis testing via social learning

Learning distributions and hypothesis testing via social learning UMich EECS 2015 1 / 48 Learning distributions and hypothesis testing via social learning Anand D. Department of Electrical and Computer Engineering, The State University of New Jersey September 29, 2015

More information

Asymptotics, asynchrony, and asymmetry in distributed consensus

Asymptotics, asynchrony, and asymmetry in distributed consensus DANCES Seminar 1 / Asymptotics, asynchrony, and asymmetry in distributed consensus Anand D. Information Theory and Applications Center University of California, San Diego 9 March 011 Joint work with Alex

More information

Weighted Gossip: Distributed Averaging Using Non-Doubly Stochastic Matrices

Weighted Gossip: Distributed Averaging Using Non-Doubly Stochastic Matrices Weighted Gossip: Distributed Averaging Using Non-Doubly Stochastic Matrices Florence Bénézit ENS-INRIA, France Vincent Blondel UCL, Belgium Patric Thiran EPFL, Switzerland John Tsitsilis MIT, USA Martin

More information

Perron Frobenius Theory

Perron Frobenius Theory Perron Frobenius Theory Oskar Perron Georg Frobenius (1880 1975) (1849 1917) Stefan Güttel Perron Frobenius Theory 1 / 10 Positive and Nonnegative Matrices Let A, B R m n. A B if a ij b ij i, j, A > B

More information

spring, math 204 (mitchell) list of theorems 1 Linear Systems Linear Transformations Matrix Algebra

spring, math 204 (mitchell) list of theorems 1 Linear Systems Linear Transformations Matrix Algebra spring, 2016. math 204 (mitchell) list of theorems 1 Linear Systems THEOREM 1.0.1 (Theorem 1.1). Uniqueness of Reduced Row-Echelon Form THEOREM 1.0.2 (Theorem 1.2). Existence and Uniqueness Theorem THEOREM

More information

Introduction to Machine Learning CMU-10701

Introduction to Machine Learning CMU-10701 Introduction to Machine Learning CMU-10701 Markov Chain Monte Carlo Methods Barnabás Póczos & Aarti Singh Contents Markov Chain Monte Carlo Methods Goal & Motivation Sampling Rejection Importance Markov

More information

Generating random spanning trees. Andrei Broder. DEC - Systems Research Center 130 Lytton Ave., Palo Alto, CA

Generating random spanning trees. Andrei Broder. DEC - Systems Research Center 130 Lytton Ave., Palo Alto, CA Generating random spanning trees Andrei Broder DEC - Systems Research Center 30 Lytton Ave., Palo Alto, CA 9430 Extended abstract Abstract. This paper describes a probabilistic algorithm that, given a

More information

Statistics 992 Continuous-time Markov Chains Spring 2004

Statistics 992 Continuous-time Markov Chains Spring 2004 Summary Continuous-time finite-state-space Markov chains are stochastic processes that are widely used to model the process of nucleotide substitution. This chapter aims to present much of the mathematics

More information

Consensus Seeking in Multi-agent Systems Under Dynamically Changing Interaction Topologies

Consensus Seeking in Multi-agent Systems Under Dynamically Changing Interaction Topologies IEEE TRANSACTIONS ON AUTOMATIC CONTROL, SUBMITTED FOR PUBLICATION AS A TECHNICAL NOTE. 1 Consensus Seeking in Multi-agent Systems Under Dynamically Changing Interaction Topologies Wei Ren, Student Member,

More information

Math 1553, Introduction to Linear Algebra

Math 1553, Introduction to Linear Algebra Learning goals articulate what students are expected to be able to do in a course that can be measured. This course has course-level learning goals that pertain to the entire course, and section-level

More information

Bounds for (generalised) Lyapunov exponents for random products of shears

Bounds for (generalised) Lyapunov exponents for random products of shears Bounds for (generalised) Lyapunov exponents for random products of shears Rob Sturman Department of Mathematics University of Leeds LAND Seminar, 13 June 2016 Leeds Joint work with Jean-Luc Thiffeault

More information

Markov Chains and Stochastic Sampling

Markov Chains and Stochastic Sampling Part I Markov Chains and Stochastic Sampling 1 Markov Chains and Random Walks on Graphs 1.1 Structure of Finite Markov Chains We shall only consider Markov chains with a finite, but usually very large,

More information

Agreement algorithms for synchronization of clocks in nodes of stochastic networks

Agreement algorithms for synchronization of clocks in nodes of stochastic networks UDC 519.248: 62 192 Agreement algorithms for synchronization of clocks in nodes of stochastic networks L. Manita, A. Manita National Research University Higher School of Economics, Moscow Institute of

More information

Distributed Estimation and Detection for Smart Grid

Distributed Estimation and Detection for Smart Grid Distributed Estimation and Detection for Smart Grid Texas A&M University Joint Wor with: S. Kar (CMU), R. Tandon (Princeton), H. V. Poor (Princeton), and J. M. F. Moura (CMU) 1 Distributed Estimation/Detection

More information

Boolean Inner-Product Spaces and Boolean Matrices

Boolean Inner-Product Spaces and Boolean Matrices Boolean Inner-Product Spaces and Boolean Matrices Stan Gudder Department of Mathematics, University of Denver, Denver CO 80208 Frédéric Latrémolière Department of Mathematics, University of Denver, Denver

More information

Combinatorial semigroups and induced/deduced operators

Combinatorial semigroups and induced/deduced operators Combinatorial semigroups and induced/deduced operators G. Stacey Staples Department of Mathematics and Statistics Southern Illinois University Edwardsville Modified Hypercubes Particular groups & semigroups

More information

Quantum walk algorithms

Quantum walk algorithms Quantum walk algorithms Andrew Childs Institute for Quantum Computing University of Waterloo 28 September 2011 Randomized algorithms Randomness is an important tool in computer science Black-box problems

More information

P i [B k ] = lim. n=1 p(n) ii <. n=1. V i :=

P i [B k ] = lim. n=1 p(n) ii <. n=1. V i := 2.7. Recurrence and transience Consider a Markov chain {X n : n N 0 } on state space E with transition matrix P. Definition 2.7.1. A state i E is called recurrent if P i [X n = i for infinitely many n]

More information

S-adic sequences A bridge between dynamics, arithmetic, and geometry

S-adic sequences A bridge between dynamics, arithmetic, and geometry S-adic sequences A bridge between dynamics, arithmetic, and geometry J. M. Thuswaldner (joint work with P. Arnoux, V. Berthé, M. Minervino, and W. Steiner) Marseille, November 2017 PART 3 S-adic Rauzy

More information

Markov Chains CK eqns Classes Hitting times Rec./trans. Strong Markov Stat. distr. Reversibility * Markov Chains

Markov Chains CK eqns Classes Hitting times Rec./trans. Strong Markov Stat. distr. Reversibility * Markov Chains Markov Chains A random process X is a family {X t : t T } of random variables indexed by some set T. When T = {0, 1, 2,... } one speaks about a discrete-time process, for T = R or T = [0, ) one has a continuous-time

More information

http://www.math.uah.edu/stat/markov/.xhtml 1 of 9 7/16/2009 7:20 AM Virtual Laboratories > 16. Markov Chains > 1 2 3 4 5 6 7 8 9 10 11 12 1. A Markov process is a random process in which the future is

More information

CHILE LECTURES: MULTIPLICATIVE ERGODIC THEOREM, FRIENDS AND APPLICATIONS

CHILE LECTURES: MULTIPLICATIVE ERGODIC THEOREM, FRIENDS AND APPLICATIONS CHILE LECTURES: MULTIPLICATIVE ERGODIC THEOREM, FRIENDS AND APPLICATIONS. Motivation Context: (Ω, P) a probability space; σ : Ω Ω is a measure-preserving transformation P(σ A) = P(A) for all measurable

More information

On Distributed Coordination of Mobile Agents with Changing Nearest Neighbors

On Distributed Coordination of Mobile Agents with Changing Nearest Neighbors On Distributed Coordination of Mobile Agents with Changing Nearest Neighbors Ali Jadbabaie Department of Electrical and Systems Engineering University of Pennsylvania Philadelphia, PA 19104 jadbabai@seas.upenn.edu

More information

6 Markov Chain Monte Carlo (MCMC)

6 Markov Chain Monte Carlo (MCMC) 6 Markov Chain Monte Carlo (MCMC) The underlying idea in MCMC is to replace the iid samples of basic MC methods, with dependent samples from an ergodic Markov chain, whose limiting (stationary) distribution

More information

Consensus Seeking in Multi-agent Systems Under Dynamically Changing Interaction Topologies

Consensus Seeking in Multi-agent Systems Under Dynamically Changing Interaction Topologies IEEE TRANSACTIONS ON AUTOMATIC CONTROL, SUBMITTED FOR PUBLICATION AS A TECHNICAL NOTE. Consensus Seeking in Multi-agent Systems Under Dynamically Changing Interaction Topologies Wei Ren, Student Member,

More information

Alternate Characterizations of Markov Processes

Alternate Characterizations of Markov Processes Chapter 10 Alternate Characterizations of Markov Processes This lecture introduces two ways of characterizing Markov processes other than through their transition probabilities. Section 10.1 addresses

More information

Understanding MCMC. Marcel Lüthi, University of Basel. Slides based on presentation by Sandro Schönborn

Understanding MCMC. Marcel Lüthi, University of Basel. Slides based on presentation by Sandro Schönborn Understanding MCMC Marcel Lüthi, University of Basel Slides based on presentation by Sandro Schönborn 1 The big picture which satisfies detailed balance condition for p(x) an aperiodic and irreducable

More information

Introduction to Search Engine Technology Introduction to Link Structure Analysis. Ronny Lempel Yahoo Labs, Haifa

Introduction to Search Engine Technology Introduction to Link Structure Analysis. Ronny Lempel Yahoo Labs, Haifa Introduction to Search Engine Technology Introduction to Link Structure Analysis Ronny Lempel Yahoo Labs, Haifa Outline Anchor-text indexing Mathematical Background Motivation for link structure analysis

More information

Consensus of Information Under Dynamically Changing Interaction Topologies

Consensus of Information Under Dynamically Changing Interaction Topologies Consensus of Information Under Dynamically Changing Interaction Topologies Wei Ren and Randal W. Beard Abstract This paper considers the problem of information consensus among multiple agents in the presence

More information

Geometric Constraints II

Geometric Constraints II Geometric Constraints II Realizability, Rigidity and Related theorems. Embeddability of Metric Spaces Section 1 Given the matrix D d i,j 1 i,j n corresponding to a metric space, give conditions under which

More information

LIMITING PROBABILITY TRANSITION MATRIX OF A CONDENSED FIBONACCI TREE

LIMITING PROBABILITY TRANSITION MATRIX OF A CONDENSED FIBONACCI TREE International Journal of Applied Mathematics Volume 31 No. 18, 41-49 ISSN: 1311-178 (printed version); ISSN: 1314-86 (on-line version) doi: http://dx.doi.org/1.173/ijam.v31i.6 LIMITING PROBABILITY TRANSITION

More information

Linear Stochastic Systems: A Geometric Approach to Modeling, Estimation and Identification

Linear Stochastic Systems: A Geometric Approach to Modeling, Estimation and Identification page i Linear Stochastic Systems: A Geometric Approach to Modeling, Estimation and Identification Anders Lindquist and Giorgio Picci January 28, 2007 page i ii Boo page Contents Preface vii 1 Introduction

More information

Uncertainty and Randomization

Uncertainty and Randomization Uncertainty and Randomization The PageRank Computation in Google Roberto Tempo IEIIT-CNR Politecnico di Torino tempo@polito.it 1993: Robustness of Linear Systems 1993: Robustness of Linear Systems 16 Years

More information

6.842 Randomness and Computation February 24, Lecture 6

6.842 Randomness and Computation February 24, Lecture 6 6.8 Randomness and Computation February, Lecture 6 Lecturer: Ronitt Rubinfeld Scribe: Mutaamba Maasha Outline Random Walks Markov Chains Stationary Distributions Hitting, Cover, Commute times Markov Chains

More information

Section 1.7: Properties of the Leslie Matrix

Section 1.7: Properties of the Leslie Matrix Section 1.7: Properties of the Leslie Matrix Definition: A matrix A whose entries are nonnegative (positive) is called a nonnegative (positive) matrix, denoted as A 0 (A > 0). Definition: A square m m

More information

Quantized average consensus via dynamic coding/decoding schemes

Quantized average consensus via dynamic coding/decoding schemes Proceedings of the 47th IEEE Conference on Decision and Control Cancun, Mexico, Dec 9-, 2008 Quantized average consensus via dynamic coding/decoding schemes Ruggero Carli Francesco Bullo Sandro Zampieri

More information

Convergence speed in distributed consensus and averaging

Convergence speed in distributed consensus and averaging Convergence speed in distributed consensus and averaging The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation As Published Publisher

More information

Convergence Rate for Consensus with Delays

Convergence Rate for Consensus with Delays Convergence Rate for Consensus with Delays Angelia Nedić and Asuman Ozdaglar October 8, 2007 Abstract We study the problem of reaching a consensus in the values of a distributed system of agents with time-varying

More information

Consensus Problems on Small World Graphs: A Structural Study

Consensus Problems on Small World Graphs: A Structural Study Consensus Problems on Small World Graphs: A Structural Study Pedram Hovareshti and John S. Baras 1 Department of Electrical and Computer Engineering and the Institute for Systems Research, University of

More information

On the mathematical background of Google PageRank algorithm

On the mathematical background of Google PageRank algorithm Working Paper Series Department of Economics University of Verona On the mathematical background of Google PageRank algorithm Alberto Peretti, Alberto Roveda WP Number: 25 December 2014 ISSN: 2036-2919

More information

Lyapunov exponents of Teichmüller flows

Lyapunov exponents of Teichmüller flows Lyapunov exponents ofteichmüller flows p 1/6 Lyapunov exponents of Teichmüller flows Marcelo Viana IMPA - Rio de Janeiro Lyapunov exponents ofteichmüller flows p 2/6 Lecture # 1 Geodesic flows on translation

More information

Lecture 5: Random Walks and Markov Chain

Lecture 5: Random Walks and Markov Chain Spectral Graph Theory and Applications WS 20/202 Lecture 5: Random Walks and Markov Chain Lecturer: Thomas Sauerwald & He Sun Introduction to Markov Chains Definition 5.. A sequence of random variables

More information

Copositive matrices and periodic dynamical systems

Copositive matrices and periodic dynamical systems Extreme copositive matrices and periodic dynamical systems Weierstrass Institute (WIAS), Berlin Optimization without borders Dedicated to Yuri Nesterovs 60th birthday February 11, 2016 and periodic dynamical

More information

Language Acquisition and Parameters: Part II

Language Acquisition and Parameters: Part II Language Acquisition and Parameters: Part II Matilde Marcolli CS0: Mathematical and Computational Linguistics Winter 205 Transition Matrices in the Markov Chain Model absorbing states correspond to local

More information

Part I Stochastic variables and Markov chains

Part I Stochastic variables and Markov chains Part I Stochastic variables and Markov chains Random variables describe the behaviour of a phenomenon independent of any specific sample space Distribution function (cdf, cumulative distribution function)

More information

E-Companion to The Evolution of Beliefs over Signed Social Networks

E-Companion to The Evolution of Beliefs over Signed Social Networks OPERATIONS RESEARCH INFORMS E-Companion to The Evolution of Beliefs over Signed Social Networks Guodong Shi Research School of Engineering, CECS, The Australian National University, Canberra ACT 000, Australia

More information

MATH 56A: STOCHASTIC PROCESSES CHAPTER 1

MATH 56A: STOCHASTIC PROCESSES CHAPTER 1 MATH 56A: STOCHASTIC PROCESSES CHAPTER. Finite Markov chains For the sake of completeness of these notes I decided to write a summary of the basic concepts of finite Markov chains. The topics in this chapter

More information

Math Homework 5 Solutions

Math Homework 5 Solutions Math 45 - Homework 5 Solutions. Exercise.3., textbook. The stochastic matrix for the gambler problem has the following form, where the states are ordered as (,, 4, 6, 8, ): P = The corresponding diagram

More information

Convex Optimization of Graph Laplacian Eigenvalues

Convex Optimization of Graph Laplacian Eigenvalues Convex Optimization of Graph Laplacian Eigenvalues Stephen Boyd Abstract. We consider the problem of choosing the edge weights of an undirected graph so as to maximize or minimize some function of the

More information

Differential Topology Final Exam With Solutions

Differential Topology Final Exam With Solutions Differential Topology Final Exam With Solutions Instructor: W. D. Gillam Date: Friday, May 20, 2016, 13:00 (1) Let X be a subset of R n, Y a subset of R m. Give the definitions of... (a) smooth function

More information

Ergodic Theorems. Samy Tindel. Purdue University. Probability Theory 2 - MA 539. Taken from Probability: Theory and examples by R.

Ergodic Theorems. Samy Tindel. Purdue University. Probability Theory 2 - MA 539. Taken from Probability: Theory and examples by R. Ergodic Theorems Samy Tindel Purdue University Probability Theory 2 - MA 539 Taken from Probability: Theory and examples by R. Durrett Samy T. Ergodic theorems Probability Theory 1 / 92 Outline 1 Definitions

More information

Average laws in analysis

Average laws in analysis Average laws in analysis Silvius Klein Norwegian University of Science and Technology (NTNU) The law of large numbers: informal statement The theoretical expected value of an experiment is approximated

More information

Powerful tool for sampling from complicated distributions. Many use Markov chains to model events that arise in nature.

Powerful tool for sampling from complicated distributions. Many use Markov chains to model events that arise in nature. Markov Chains Markov chains: 2SAT: Powerful tool for sampling from complicated distributions rely only on local moves to explore state space. Many use Markov chains to model events that arise in nature.

More information

Markov processes and queueing networks

Markov processes and queueing networks Inria September 22, 2015 Outline Poisson processes Markov jump processes Some queueing networks The Poisson distribution (Siméon-Denis Poisson, 1781-1840) { } e λ λ n n! As prevalent as Gaussian distribution

More information

On Backward Product of Stochastic Matrices

On Backward Product of Stochastic Matrices On Backward Product of Stochastic Matrices Behrouz Touri and Angelia Nedić 1 Abstract We study the ergodicity of backward product of stochastic and doubly stochastic matrices by introducing the concept

More information

MATH 56A: STOCHASTIC PROCESSES CHAPTER 2

MATH 56A: STOCHASTIC PROCESSES CHAPTER 2 MATH 56A: STOCHASTIC PROCESSES CHAPTER 2 2. Countable Markov Chains I started Chapter 2 which talks about Markov chains with a countably infinite number of states. I did my favorite example which is on

More information

No class on Thursday, October 1. No office hours on Tuesday, September 29 and Thursday, October 1.

No class on Thursday, October 1. No office hours on Tuesday, September 29 and Thursday, October 1. Stationary Distributions Monday, September 28, 2015 2:02 PM No class on Thursday, October 1. No office hours on Tuesday, September 29 and Thursday, October 1. Homework 1 due Friday, October 2 at 5 PM strongly

More information

Non-Essential Uses of Probability in Analysis Part IV Efficient Markovian Couplings. Krzysztof Burdzy University of Washington

Non-Essential Uses of Probability in Analysis Part IV Efficient Markovian Couplings. Krzysztof Burdzy University of Washington Non-Essential Uses of Probability in Analysis Part IV Efficient Markovian Couplings Krzysztof Burdzy University of Washington 1 Review See B and Kendall (2000) for more details. See also the unpublished

More information

The Theory behind PageRank

The Theory behind PageRank The Theory behind PageRank Mauro Sozio Telecom ParisTech May 21, 2014 Mauro Sozio (LTCI TPT) The Theory behind PageRank May 21, 2014 1 / 19 A Crash Course on Discrete Probability Events and Probability

More information

Mathematical foundations - linear algebra

Mathematical foundations - linear algebra Mathematical foundations - linear algebra Andrea Passerini passerini@disi.unitn.it Machine Learning Vector space Definition (over reals) A set X is called a vector space over IR if addition and scalar

More information

Parametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes

Parametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes Parametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes Electrical & Computer Engineering North Carolina State University Acknowledgment: ECE792-41 slides were adapted

More information

A Relationship Between Minimum Bending Energy and Degree Elevation for Bézier Curves

A Relationship Between Minimum Bending Energy and Degree Elevation for Bézier Curves A Relationship Between Minimum Bending Energy and Degree Elevation for Bézier Curves David Eberly, Geometric Tools, Redmond WA 9852 https://www.geometrictools.com/ This work is licensed under the Creative

More information

Reaching a Consensus in a Dynamically Changing Environment A Graphical Approach

Reaching a Consensus in a Dynamically Changing Environment A Graphical Approach Reaching a Consensus in a Dynamically Changing Environment A Graphical Approach M. Cao Yale Univesity A. S. Morse Yale University B. D. O. Anderson Australia National University and National ICT Australia

More information

Propp-Wilson Algorithm (and sampling the Ising model)

Propp-Wilson Algorithm (and sampling the Ising model) Propp-Wilson Algorithm (and sampling the Ising model) Danny Leshem, Nov 2009 References: Haggstrom, O. (2002) Finite Markov Chains and Algorithmic Applications, ch. 10-11 Propp, J. & Wilson, D. (1996)

More information

Advanced Computer Networks Lecture 2. Markov Processes

Advanced Computer Networks Lecture 2. Markov Processes Advanced Computer Networks Lecture 2. Markov Processes Husheng Li Min Kao Department of Electrical Engineering and Computer Science University of Tennessee, Knoxville Spring, 2016 1/28 Outline 2/28 1 Definition

More information