Some new randomized methods for control and optimization

Transcription

1 Some new randomized methods for control and optimization B.T. Polyak Institute for Control Sciences, Russian Academy of Sciences, Moscow, Russia June 29 with E. Gryazina, P. Scherbakov Workshop in honor of B.Barmish for the occasion of his 6th birthday B.T. Polyak (ICS, Moscow) Randomized methods June 29 1 / 41

2 CONGRATULATIONS! Bob, it s been a long time we have worked together, but I remember this time with delight. I remember your eyes shining when we discovered e E picture for randomized robustness... Remain as young and vital as ever! B.T. Polyak (ICS, Moscow) Randomized methods June 29 2 / 41

3 Outline Historical remarks 2 problems generation of points in a set and optimization Ideal Monte Carlo and convergence estimates Markov Chain Monte Carlo Boundary oracle Hit-and-Run Shake-and-Bake Exploiting barriers Applications to control Applications to optimization Conclusions B.T. Polyak (ICS, Moscow) Randomized methods June 29 3 / 41

4 Historical remarks First random search methods for optimization Rastrigin (196-ies) Randomized methods for control Stengel and Ray (199) Barmish and Polyak (1996) Tempo, Calafiore, Dabbene (24) Revival of randomized approaches for optimization Bertsimas and Vempala (24), Dabbene, Shcherbakov and Polyak (28) B.T. Polyak (ICS, Moscow) Randomized methods June 29 4 / 41

5 Historical remarks First random search methods for optimization Rastrigin (196-ies) Randomized methods for control Stengel and Ray (199) Barmish and Polyak (1996) Tempo, Calafiore, Dabbene (24) Revival of randomized approaches for optimization Bertsimas and Vempala (24), Dabbene, Shcherbakov and Polyak (28) B.T. Polyak (ICS, Moscow) Randomized methods June 29 4 / 41

6 Historical remarks First random search methods for optimization Rastrigin (196-ies) Randomized methods for control Stengel and Ray (199) Barmish and Polyak (1996) Tempo, Calafiore, Dabbene (24) Revival of randomized approaches for optimization Bertsimas and Vempala (24), Dabbene, Shcherbakov and Polyak (28) B.T. Polyak (ICS, Moscow) Randomized methods June 29 4 / 41

7 2 main problems 1. Generation of points in a set X R n. It can be nonconvex and non-connected. Applications: generate stabilizing controllers and calculate performance specifications, generate perturbations and check robustness. 2. Convex optimization min c T x s.t. x X X is a convex bounded closed set in R n with nonempty interior. B.T. Polyak (ICS, Moscow) Randomized methods June 29 5 / 41

8 Ideal Monte Carlo x 1, x 2,... x N independent uniformly distributed points in X Cutting plane method for optimization 1. Set X 1 = X 2. Generate uniform x 1, x 2,... x N X k 3. Find f k = min c T x i 4. Set X k+1 = X k {x : c T x f k } go to Step 2. B.T. Polyak (ICS, Moscow) Randomized methods June 29 6 / 41

9 Convergence estimates f = max x X ct x, f = min x X ct x, h = f f Theorem After k iterations of the algorithm E [f k ] f q k, q = h ( n B N + 1, 1 ), n where B(a, b) is Euler beta-function. Dabbene, Scherbakov, Polyak, 47th CDC, 28 B.T. Polyak (ICS, Moscow) Randomized methods June 29 7 / 41

10 Radon theorem and center of gravity method Case of a special interest: N = 1, k = 1 Theorem Let x 1 be a random point uniformly distributed in X. Then E [ ( c T ] x 1 f h 1 1 ). n + 1 ( E [x 1 ] = g (center of gravity of X), c T g f h 1 1 ) n + 1 [Radon theorem (1916)] center of gravity method x k = g k, X k+1 = X k {x : c T x c T g k } B.T. Polyak (ICS, Moscow) Randomized methods June 29 8 / 41

11 Implementable random algorithms: MCMC How to implement Monte Carlo method? X is a simple set (box, ball, simplex etc.) Rejection method Markov-Chain Monte Carlo schemes (random walks in X) P.Diaconis The Markov chain Monte Carlo revolution, Bull. of the AMS, 29, Vol. 46. No 2. pp B.T. Polyak (ICS, Moscow) Randomized methods June 29 9 / 41

12 Implementable random algorithms: MCMC How to implement Monte Carlo method? X is a simple set (box, ball, simplex etc.) Rejection method Markov-Chain Monte Carlo schemes (random walks in X) P.Diaconis The Markov chain Monte Carlo revolution, Bull. of the AMS, 29, Vol. 46. No 2. pp B.T. Polyak (ICS, Moscow) Randomized methods June 29 9 / 41

13 Implementable random algorithms: MCMC How to implement Monte Carlo method? X is a simple set (box, ball, simplex etc.) Rejection method Markov-Chain Monte Carlo schemes (random walks in X) P.Diaconis The Markov chain Monte Carlo revolution, Bull. of the AMS, 29, Vol. 46. No 2. pp B.T. Polyak (ICS, Moscow) Randomized methods June 29 9 / 41

14 Implementable random algorithms: MCMC How to implement Monte Carlo method? X is a simple set (box, ball, simplex etc.) Rejection method Markov-Chain Monte Carlo schemes (random walks in X) P.Diaconis The Markov chain Monte Carlo revolution, Bull. of the AMS, 29, Vol. 46. No 2. pp B.T. Polyak (ICS, Moscow) Randomized methods June 29 9 / 41

15 Boundary oracle Given x X, d vector specifying the direction in R n L = {t R : x + td X} Boundary oracle For convex sets L = (t, t), where t = inf{t : x + td X}, t = sup{t : x + td X} Complete boundary oracle L = {t R : x + td X} + inner normals to X at the boundary points B.T. Polyak (ICS, Moscow) Randomized methods June 29 1 / 41

16 Boundary oracle Given x X, d vector specifying the direction in R n L = {t R : x + td X} Boundary oracle For convex sets L = (t, t), where t = inf{t : x + td X}, t = sup{t : x + td X} Complete boundary oracle L = {t R : x + td X} + inner normals to X at the boundary points B.T. Polyak (ICS, Moscow) Randomized methods June 29 1 / 41

17 Boundary oracle is available for numerous sets LMI set { } n X = x R n : A + x i A i LMI constrained set of symmetric matrices P X = { P : AP + P A T + C, P } Quadratic matrix inequalities set X = { P : AP + P A T + P BB T P + C, P } Linear algebraic inequalities set X = { x R n : c T i x a i, i = 1,..., m } i=1 B.T. Polyak (ICS, Moscow) Randomized methods June / 41

18 Hit-and-Run 1. i =, x X 2. Choose random direction d uniformly distributed on the unit sphere 3. x i+1 = x i + t 1 d, t 1 is uniformly distributed on L = (t, t) 4. L is updated with respect to x i+1, go to Step 2. Theorem (Turchin (1971), Smith (1984)) Let X be bounded open or coincides with the closure of interior points of X. Then for any measurable set A X probability P i (A) = P (x i A x ) can be estimated as P i (A) P (A) q i, where P (A) = Vol(A) Vol(X) and q < 1 does not depend on x. B.T. Polyak (ICS, Moscow) Randomized methods June / 41

19 Shake-and-Bake: an alternative way for generating points Points are asymptotically uniformly distributed in the boundary of X. Complete boundary oracle is exploited. LMI set { } n X = x R n : A + x i A i n i = (A i e, e), where e is the eigenvector corresponding to n zero eigenvalue of the matrix A + x i A i. i=1 LMI constrained set of symmetric matrices P X = { P : AP + P A T + C } N = (ee T A + A T ee T ), i=1 where e is the eigenvector corresponding to zero eigenvalue of the matrix AP + P A T + C. B.T. Polyak (ICS, Moscow) Randomized methods June / 41

20 Shake-and-Bake: the algorithm 1. i =, x X, n is the normal. 2. Choose random direction s i, s i = 1 ξ 2 n 1 n + r, ξ uniform random in (, 1), r is random unit uniform direction (n, r) =. 3. x i+1 = x i + ts, t is given by the boundary oracle for the direction s. 4. L is updated with respect to x i+1, go to Step 2. B.T. Polyak (ICS, Moscow) Randomized methods June / 41

21 Shake-and-Bake for nonconvex sets B.T. Polyak (ICS, Moscow) Randomized methods June / 41

22 Numerical simulation Standard SDP of the form min c T x n s.t. A + x i A i i=1 A i, i =, 1,... n symmetric real matrices m m; c = [,...,, 1] We applied modified HR where min x i was replaced with averaged X i + various heuristic acceleration methods (scaling, projecting, accelerating step) Open problem: number of HR points in every step. B.T. Polyak (ICS, Moscow) Randomized methods June / 41

23 Disadvantages of HR Jams in a corner. Jams for long or thin bodies. (Lovasz, Vempala. Hit-and-Run from a corner, 27) Number of iterations to achieve accuracy ε N > 1 1 n2 R 2 r 2 ln M ε How to accelerate? Smith (1998) d uniformly on a sphere another distribution H B.T. Polyak (ICS, Moscow) Randomized methods June / 41

24 y Barrier functions in convex optimization Yu. Nesterov, A. Nemirovski, S. Boyd Convex function F (x) is a barrier for a convex set X, if F (x) is defined on interior of X and F (x) for x X. Self-concordant barriers, Newton method Interior-point methods are highly effective for convex optimization! x B.T. Polyak (ICS, Moscow) Randomized methods June / 41

25 Modified Hit-and-Run by use of barriers F (x) self-concordant barrier for X F (x + y) = F (x ) + ( F (x ), y) ( 2 F (x )y, y) +o(y 2 ), }{{} F (y) F (x) quadratic approximation of F (x) at x. Dikin s ellipsoid E = {y : ( 2 F (x )(y x ), y x ) 1} X B.T. Polyak (ICS, Moscow) Randomized methods June / 41

26 Modified Hit-and-Run by use of barriers contd Random direction d = ( 2 F ) 1/2 ξ, ξ uniformly on a sphere Next point x = x + λd, λ [λ, λ] as in Hit-and-Run B.T. Polyak (ICS, Moscow) Randomized methods June 29 2 / 41

27 Modified Hit-and-Run by use of barriers: example Q = {x R 2 : x i a i }, a = [1 4, 1] strip 1 points in R Magenta Hit-and-Run.6 x Blue Modified Hit-and-Run x 1 x 1 5 B.T. Polyak (ICS, Moscow) Randomized methods June / 41

28 y Barrier MC: Phase 1 F (x) self-concordant barrier for X F (x + y) = F (x ) + ( F (x ), y) ( 2 F (x )y, y) +o(y 2 ), }{{} F (y) E = {y : F (y) }, or E = {y : ( 2 F (x )(y x ), y x )) δ}, x = ( 2 F (x )) 1 F (x ), δ = (( 2 F (x )) 1 F (x ), F (x )) Random direction 1 ( d = x 2 ) 1/2 F + ξ, δ 1 2 x * x ξ uniformly on a sphere 3 4 B.T. Polyak (ICS, Moscow) Randomized methods June / x

29 Advantages Departs corners. Well walks in long and thin sets. Conjecture x k asymptotically distributed with density ρ(x) > on X. Phase ( 2. 2 ) 1/2 F Ellipsoid is defined by matrix H i =. δ After N iterations let H = 1 Hi and generate random direction N d = Hξ, ξ uniformly on unit sphere B.T. Polyak (ICS, Moscow) Randomized methods June / 41

30 Barrier MC can be applied for Polyhedral sets in R n X = {x R n : (a i, x) b i, i = 1,..., m}. m F (x) = ln(b i (a i, x)), F (x ) = a i 1 (a i, x ), i=1 2 F (x ) = a i a it (1 (a i, x )) 2 LMI in standard format X = {x R l : A(x) = A + F (x) = ln det(a(x)) LMI with matrix variables l x i A i, A i S n n }. i=1 Q = {X, trcx 1, X, C S n n }. F (X) = ln detx ln(1 trcx) B.T. Polyak (ICS, Moscow) Randomized methods June / 41

31 Example: diamond X = {x R n : x i, (a, x) 1}, a = [1,..., 1, 1 4 ], F (x) = lnx i ln(1 (a, x)) x B.T. Polyak (ICS, Moscow) Randomized methods June / 41

32 Comparison with pure MC X = {x R n : x i, (a, x) 1}, a = [1,..., 1, 1 4 ] f i = (a, x i ), f 1, x Q If x i are uniform on X, then cdf t Φ(t) = p(f)df t n Φ(t) n = 3 N = 2 n = 1 N = 1 n = 2 N = t B.T. Polyak (ICS, Moscow) Randomized methods June / 41

33 Empirical density (N = 5, R 2 ) 4 1 New Method Phase 1 1 x 1 4 Hit and Run New Method Phase 2 1 x 1 4 Ideal Monte Carlo B.T. Polyak (ICS, Moscow) Randomized methods June / 41

34 Example: stick and sheet R 2 Q = {x R n : x i a i } a = [1 4,..., 1 4, 1] a = [1,..., 1, 1 4 ] 1 R 2 N=1 12 x 1 5 R 2 N= x 2.5 x n x 1 x x 1 B.T. Polyak (ICS, Moscow) Randomized methods June / 41

35 Example: stick and sheet R2 Q = {x Rn : xi ai } a = [1 4,..., 1 4, 1] a = [1,..., 1, 1 4 ] R2 5 x 1 R2 N=5 N= xn xn x1 B.T. Polyak (ICS, Moscow) x Randomized methods x June / 41

36 Example: stick and sheet R 2 Phase 2 Q = {x R n : x i a i } a = [1 4,..., 1 4, 1] a = [1,..., 1, 1 4 ] R 2 ÔÀÇÀ 2 N= x 1 5 R 2 N= x n x n x x x B.T. Polyak (ICS, Moscow) Randomized methods June 29 3 / 41

37 Standard LMI l X = {x R l : A(x) = A + x i A i, A i S n n } i=1 F (x) = ln det(a(x)) F (x ) i = tra i A(x ) 1, 2 F (x ) ij = tra(x ) 1 A i A(x ) 1 A j 1 1 x A, A 1, A 2 random matrices 4 4, A 1 points x 1 B.T. Polyak (ICS, Moscow) Randomized methods June / 41

38 LMI with matrix variables Q = {X, trcx 1, X, C S n n }. F (X) = ln detx ln(1 trcx), ε = 1 trcx Ellipsoid E = {Y : F (y) } is X + X CX, Y + 1 ε 2 Y + X CY CX, Y ε 2 Center of ellipsoid Y solution of Lyapunov equation AXB T X + G = where A = X ( ) C ε,b = CX X CX, G = X 2 ε Choice of direction for ellipsoid AXA + BXB, X δ Z - uniform on Z F = 1, λ uniform on [ α, α], α : α 2 AZA + BZB, Z = δ, then λz E. B.T. Polyak (ICS, Moscow) Randomized methods June / 41

39 X : 2 2, C = I {[ ] p11 p Q = 12 p 12 p 22 }, p 11 + p 22 1 p 11 >, p 22 > p 11 p 22 p 2 12 p 11 + p P P 22 P B.T. Polyak (ICS, Moscow) Randomized methods June / 41

40 Comparison with pure MC Theorem If x i are uniform i.i.d. on X R n, f = (c, x), f = min(c, x), h = max (c, x) f then X E[min f i ] f h X B ( N + 1, 1 n n ), i = 1,..., N n N C E[min f i ] 1 E[max f i ] I diag(1,1) I diag(1,...,1,1) B.T. Polyak (ICS, Moscow) Randomized methods June / 41

41 Comparison with pure MC t Φ(t) = p(f)df t m, m = n(n + 1) B.T. Polyak (ICS, Moscow) Randomized methods June / 41

42 Applications to control Sets with available boundary oracle Stability set for polynomials K = {k R n : p(s, k) = p (s) + Stability set for matrices n k i p i (s) is stable} A R n n, B R n m, C R l n K = {K R m l : A + BKC is stable} Robust stability set for polynomials n K = {k : P (s, q) + k i P i (s, q) is stable q Q}, Q R m i=1 i=1 Quadratic stability set ẋ = Ax K = {P > : AP + P A T } B.T. Polyak (ICS, Moscow) Randomized methods June / 41

43 Stability set for polynomials K = {k R n : p(s, k) = p (s) + n k i p i (s) is stable} i=1 k K i.e. p(s, k ) is stable, d = s/ s, s = randn(n,1) random direction Boundary oracle: L = {t R : k + td K}, i.e. {t R : p(s, k ) + t d i p i (s) is stable}. D-decomposition problem for real scalar parameter t! Gryazina E. N., Polyak B. T. Stability regions in the parameter space: D-decomposition revisited //Automatica. 26. Vol. 42, No. 1, P B.T. Polyak (ICS, Moscow) Randomized methods June / 41

44 Example: Generating points in the disconnected set K = {k R n : p(s, k) = p (s) + n k i p i (s) is stable}, p(s, k) = 2.2s s s k 1 (s 3 + s 2 s 1) + k 2 (s 3 3s 2 + 3s 1) i=1 B.T. Polyak (ICS, Moscow) Randomized methods June / 41

45 Stability set for matrices ẋ = Ax + Bu, y = Cx, u = Ky A R n n, B R n m, C R l n ; K = {K R m l : A + BKC is stable} K K, i.e. A + BK C is stable D = Y/ Y, Y = randn(m, l) random direction in the matrix space K A + B(K + td)c = F + tg, where F = A + BK C, G = BDC Boundary oracle: L = {t R : F + tg is stable} Total description of L is hard: f(t) = max Re eig(f + tg) numerical solution of the equation f(t) =, t > (MatLab command fsolve) B.T. Polyak (ICS, Moscow) Randomized methods June / 41

46 Quadratic stability ẋ = Ax + Bu, u = Kx K = {K : P >, A T c P + P A c }, K is convex and bounded. A c = A + BK Q = P 1 >, QA T + AQ + BY + Y T B T <, Y = KQ. k K, Q = P 1, Y = K Q starting points Q = Q + tj, Y = Y + tg, where J and G are random directions in the matrix space. initial inequality F + tr < Boundary oracle: L = ( t, t), where t = min λ i, t = min µ i ; λ i real positive eigenvalues for the pair of matrices F = Q A T + AQ + BY + Y T B T and R = JA T + AJ + BG + G T B T ; µ i correspondingly for matrices F, R. B.T. Polyak (ICS, Moscow) Randomized methods June 29 4 / 41

47 Conclusions New versions of MCMC are effective Randomized approaches for optimization are promising. Proposed methods are simple in implementation and give an opportunity to solve large-dimensional problems. B.T. Polyak (ICS, Moscow) Randomized methods June / 41