On the Solution of the GPS Localization and Circle Fitting Problems

Size: px

Start display at page:

Download "On the Solution of the GPS Localization and Circle Fitting Problems"

Randall King
6 years ago
Views:

1 On the Solution of the GPS Localization and Circle Fitting Problems Amir Beck Technion - Israel Institute of Technology Haifa, Israel Joint work with Dror Pan, Technion. Applications of Optimization in Science and Engineering Institute of Pure and Applied Mathematics (IPAM), Los Angeles, Nov. 30- Dec. 3

2 The Global Positioning System a system of 31 transmitting satellites (originally 24). objective: provide reliable location and time information anywhere on Earth where there is unobstructed line of sight to four or more satellites.

3 Localization via GPS Each satellite transmits its location (x i, y i, z i ) and a time stamp t i. The GPS receiver estimates the distances to at least 4 satellites via d i = c(t t i ) c - speed of light. T - GPS receiver s clock. The receiver s clock is inaccurate (an error of one microsecond corresponds to an error or 300 meters). The measured distances are called pesudoranges and include an unknown clock bias.

4 The GPS Localization Problem d i x a i r, i = 1,..., m a i - satellite s location. r - unknown bias. x - unknown user s location. d i - i-th pseudorange (can even be negative) m n + 1.

5 The GPS Localization Problem d i x a i r, i = 1,..., m a i - satellite s location. r - unknown bias. x - unknown user s location. d i - i-th pseudorange (can even be negative) m n + 1. A Least Squares Problem: { m } ( x a i d i r) 2. min x,r

6 The GPS Localization Problem: without loss of generality: d i 0 (otherwise make a shift and redefine r). a mild assumption: r 0. The GPS Least Squares Problem { m } ( x a i d i r) 2 : r 0. min x,r

7 The GPS Localization Problem: without loss of generality: d i 0 (otherwise make a shift and redefine r). a mild assumption: r 0. The GPS Least Squares Problem { m } ( x a i d i r) 2 : r 0. min x,r Problem Reduction (minimizing with respect to r): The GPS Least Squares Problem-reduced form { m } (GPS-LS) : min ( x a i d i r(x)) 2 x " # 1 mx r(x) := ( x a i d i) m +

8 The Bad News... The GPS-LS problem is nonsmooth and nonconvex

9 The Bad News... The GPS-LS problem is nonsmooth and nonconvex In principal, it should be difficult to find a global optimal solution

10 The Circle Fitting Problem - d i = 0 When d i = 0, the problem reduces circle fitting: Given m points a 1,..., a m, find the circle that fits them in the best way. x a i r

11 The Circle Fitting Least Squares Problem The CF Least Squares Problem { m } ( x a i r) 2 : r 0. min x,r

12 The Circle Fitting Least Squares Problem The CF Least Squares Problem { m } ( x a i r) 2 : r 0. min x,r The CF Least Squares Problem-reduced form { m } ( (CF-LS) : min ( x a i r(x)) 2 r(x) := 1 x m ) m x a i

13 The Circle Fitting Least Squares Problem The CF Least Squares Problem { m } ( x a i r) 2 : r 0. min x,r The CF Least Squares Problem-reduced form { m } ( (CF-LS) : min ( x a i r(x)) 2 r(x) := 1 x m ) m x a i (CF-LS) is the geometric fitting problem: find the circle that minimizes the distances between the circle and the given points

14 Applications of Circle Fitting Archaeology Computer graphics Coordinate metrology Petroleum engineering Quality inspection for mechanical parts Statistics

15 Literature GPS Localization: Abel (1994) - A variable projection method. Source localization from range-differences (usage of a reference measurement): Huang, Benesty, Elko and Mersereau (2001), Stoica and Li (2006), Beck, Stoica and Li (2008), Sensor network localization from range difference (2009), Yang, Wang,Luo (usage of all differences, SDR approach). Circle Fitting: Kasa (1976) - solution of a related squared least squares problem in the 2D case. Gander, Golub and Strebel (1994): algebraic fit + Gauss Newton for (CF-LS). Chernov, Lesort (2005) - Analysis in the 2D case.

16 The least squares GPS localization problem Advantage: Has a statistical and geometrical meaning. Disadvantage Nonconvex and nonsmooth - seems to be intractable.

17 The least squares GPS localization problem Advantage: Has a statistical and geometrical meaning. Disadvantage Nonconvex and nonsmooth - seems to be intractable. It is therefore important to find a good approximate solution/solution to an approximate problem

18 The Squared Least Squares Approach replace x a i r + d i with x a i 2 (r + d i ) 2 remove the constraint r 0

19 The Squared Least Squares Approach replace x a i r + d i with x a i 2 (r + d i ) 2 remove the constraint r 0 The Squared Least Squares GPS problem: { m } ( (GPS-SLS): min x ai 2 (r + d i ) 2) 2. Disadvantage: loses the statistical/geomretrical meaning of LS. Advantage: tractable!! (although quartic)

20 Equivalence to GTRS (GPS-SLS): min { m } ( x ai 2 (r + d i ) 2) 2.

21 Equivalence to GTRS min x,r,α (GPS-SLS): { m min { m } ( x ai 2 (r + d i ) 2) 2. } ( 2a T i x 2d i r + α + a i 2 di 2 ) 2 : α = x 2 r 2.

22 Equivalence to GTRS min x,r,α Lemma (GPS-SLS): { m min { m } ( x ai 2 (r + d i ) 2) 2. } ( 2a T i x 2d i r + α + a i 2 di 2 ) 2 : α = x 2 r 2. Problem (GPS-SLS) is equivalent to { min By b 2 : y T Dy 2g T y = 0 }, y R n+2 0 2a T d a 1 2 d 2 1 2a T d 2 x I n 0 n 0 n B = C.. A, y αa, D 0 T n 0 0 A a 2 2 d2 2, b = B C r 0 2a T n 0 1. A m 1 2d m a m 2 dm 2

23 Tractability of GTRS Problems Generalized Trust Region Subproblem (GTRS): (GTRS): min{x T A 1 x + 2b T 1 x + c 1 : x T A 2 x + 2b T 2 x + c 2 = 0},

24 Tractability of GTRS Problems Generalized Trust Region Subproblem (GTRS): (GTRS): min{x T A 1 x + 2b T 1 x + c 1 : x T A 2 x + 2b T 2 x + c 2 = 0}, Theorem (More, 93) Suppose that A 2 0. Then x is an optimal solution of (GTRS) if and only if there exists λ R such that (A 1 + λa 2 )x + (b 1 + λb 2 ) = 0, x T A 2 x + 2b T 2 x + c 2 = 0, A 1 + λa 2 0,

25 Tractability of GTRS Problems Generalized Trust Region Subproblem (GTRS): (GTRS): min{x T A 1 x + 2b T 1 x + c 1 : x T A 2 x + 2b T 2 x + c 2 = 0}, Theorem (More, 93) Suppose that A 2 0. Then x is an optimal solution of (GTRS) if and only if there exists λ R such that (A 1 + λa 2 )x + (b 1 + λb 2 ) = 0, x T A 2 x + 2b T 2 x + c 2 = 0, A 1 + λa 2 0, The problem can be solved by a dual approach via a one-dimensional search.

26 Tractability of the GTRS problem, Cont d The global optimal solution of (GPS-SLS) is comprised of the first n components of the vector where λ is the root of y(λ ) = (B T B + λ D) 1 (B T b + λ g), over a predefined interval [µ 1, µ 2 ]. φ(λ) y(λ) T Dy(λ) 2g T y(λ) = 0,

27 Existence of an Optimal Solution of (GPS-SLS) Under what assumption does (GPS-SLS) attains an optimal solution?

28 Existence of an Optimal Solution of (GPS-SLS) Under what assumption does (GPS-SLS) attains an optimal solution? Assumption The Basic Assumption: The matrix Ã defined by 2a T 1 1 2a T 2 1 Ã =.. Rm (n+1) 1 2a T m has full column rank. That is, a 1,..., a m do not reside in a lower-dimensional space.

29 Existence of an Optimal Solution of (GPS-SLS) Under what assumption does (GPS-SLS) attains an optimal solution? Assumption The Basic Assumption: The matrix Ã defined by 2a T 1 1 2a T 2 1 Ã =.. Rm (n+1) 1 2a T m has full column rank. That is, a 1,..., a m do not reside in a lower-dimensional space. Rather mild when m n + 1.

30 Existence of an Optimal Solution of (GPS-SLS) Under what assumption does (GPS-SLS) attains an optimal solution? Assumption The Basic Assumption: The matrix Ã defined by 2a T 1 1 2a T 2 1 Ã =.. Rm (n+1) 1 2a T m has full column rank. That is, a 1,..., a m do not reside in a lower-dimensional space. Rather mild when m n + 1. Further assumptions?

31 Existence of an Optimal Solution of (GPS-SLS) (GTRS) : { min By b 2 : y T Dy 2g T y = 0 }, y R n+2 0 2a T d d 1 2a T 2 1 2d 2 I n 0 n 0 n B = C = (Ã, 2d), D 0 T n 0 0 A d 2, d = B.. A 0 C 2a T n 0 1. A. m 1 2d m d m Theorem The minimum of the GTRS problem is attained if at least one of the following conditions is satisfied: i. d / Range(Ã) [(ÃT ] ii. d Range(Ã) and Ã) 1 Ã T d 1 2. n.

32 Existence of an Optimal Solution of (GPS-SLS) (GTRS) : { min By b 2 : y T Dy 2g T y = 0 }, y R n+2 0 2a T d d 1 2a T 2 1 2d 2 I n 0 n 0 n B = C = (Ã, 2d), D 0 T n 0 0 A d 2, d = B.. A 0 C 2a T n 0 1. A. m 1 2d m d m Theorem The minimum of the GTRS problem is attained if at least one of the following conditions is satisfied: i. d / Range(Ã) mild when m n + 2, impossible when m = n + 1. [(ÃT ] ii. d Range(Ã) and Ã) 1 Ã T d 1 2. n

33 Existence of an Optimal Solution of (GPS-SLS) (GTRS) : { min By b 2 : y T Dy 2g T y = 0 }, y R n+2 0 2a T d d 1 2a T 2 1 2d 2 I n 0 n 0 n B = C = (Ã, 2d), D 0 T n 0 0 A d 2, d = B.. A 0 C 2a T n 0 1. A. m 1 2d m d m Theorem The minimum of the GTRS problem is attained if at least one of the following conditions is satisfied: i. d / Range(Ã) mild when m n + 2, impossible when m = n + 1. [(ÃT ] ii. d Range(Ã) and Ã) 1 Ã T d 1 n 2. mild when m = n + 1

34 Proof Layout (GTRS) : { min By b 2 : y T Dy 2g T y = 0 }, y R n+2 i. B = (Ã, 2d) B with full column rank obj. function coercive.

35 Proof Layout (GTRS) : { min By b 2 : y T Dy 2g T y = 0 }, y R n+2 i. B = (Ã, 2d) B with full column rank obj. function coercive. ii. A known sufficient condition: λ R : B T B + λd 0.

36 Proof Layout (GTRS) : { min By b 2 : y T Dy 2g T y = 0 }, y R n+2 i. B = (Ã, 2d) B with full column rank obj. function coercive. ii. A known sufficient condition: λ R : B T B + λd 0. (ÃT ) ( ) Ã + λe 2ÃT d In 0 2d T Ã 4 d 2 0, E = n λ 0 T. n 0

37 Proof Layout (GTRS) : { min By b 2 : y T Dy 2g T y = 0 }, y R n+2 i. B = (Ã, 2d) B with full column rank obj. function coercive. ii. A known sufficient condition: λ R : B T B + λd 0. (ÃT ) ( ) Ã + λe 2ÃT d In 0 2d T Ã 4 d 2 0, E = n λ 0 T. n 0 +Schur complement λ R : g(λ) := 4 d 2 λ 4d T Ã(Ã T Ã + λe) 1 Ã T d > 0

38 Proof Layout (GTRS) : { min By b 2 : y T Dy 2g T y = 0 }, y R n+2 i. B = (Ã, 2d) B with full column rank obj. function coercive. ii. A known sufficient condition: λ R : B T B + λd 0. (ÃT ) ( ) Ã + λe 2ÃT d In 0 2d T Ã 4 d 2 0, E = n λ 0 T. n 0 +Schur complement λ R : g(λ) := 4 d 2 λ 4d T Ã(Ã T Ã + λe) 1 Ã T d > 0 + g(0) = 0 it is enough to prove that g (0) 0

39 Proof Layout (GTRS) : { min By b 2 : y T Dy 2g T y = 0 }, y R n+2 i. B = (Ã, 2d) B with full column rank obj. function coercive. ii. A known sufficient condition: λ R : B T B + λd 0. (ÃT ) ( ) Ã + λe 2ÃT d In 0 2d T Ã 4 d 2 0, E = n λ 0 T. n 0 +Schur complement λ R : g(λ) := 4 d 2 λ 4d T Ã(Ã T Ã + λe) 1 Ã T d > 0 + g(0) = 0 it is enough to prove that g (0) 0 Final step: [ ] g (0) 0 (Ã T Ã) 1 Ã T d 1/2. n

40 The Circle Fitting SLS Problem (CF-SLS) : min x,r { m } ( x a i 2 r 2 ) 2 : x R n, r R.

41 The Circle Fitting SLS Problem (CF-SLS) : A stronger result: Theorem min x,r { m } ( x a i 2 r 2 ) 2 : x R n, r R. (CF-SLS) is equivalent to the linear least squares problem: min Ãy b 2 where b = ( a 1 2,..., a m 2 ) T. (2D case - Kasa 1976)

42 The Circle Fitting SLS Problem (CF-SLS) : A stronger result: Theorem min x,r { m } ( x a i 2 r 2 ) 2 : x R n, r R. (CF-SLS) is equivalent to the linear least squares problem: min Ãy b 2 where b = ( a 1 2,..., a m 2 ) T. (2D case - Kasa 1976) Proof Idea: m min ( 2a T i x + x 2 r 2 + a i 2 ) 2 : x R n, r R x,r }{{}. R +possible to discard the relation between R, x and r

43 So far... Main objective: solution of the nonsmooth/nonconvex GPS-LS problem: { m } (GPS-LS) : min ( x a i d i r(x)) 2 x

44 So far... Main objective: solution of the nonsmooth/nonconvex GPS-LS problem: { m } (GPS-LS) : min ( x a i d i r(x)) 2 x The related GPS-SLS problem: { m } ( (GPS-SLS): min x ai 2 (r + d i ) 2) 2. is tractable (equivalent to GTRS, or least squares)

45 So far... Main objective: solution of the nonsmooth/nonconvex GPS-LS problem: { m } (GPS-LS) : min ( x a i d i r(x)) 2 x The related GPS-SLS problem: { m } ( (GPS-SLS): min x ai 2 (r + d i ) 2) 2. is tractable (equivalent to GTRS, or least squares) Attainability of the GPS-SLS under rather mild conditions.

46 So far... Main objective: solution of the nonsmooth/nonconvex GPS-LS problem: { m } (GPS-LS) : min ( x a i d i r(x)) 2 x The related GPS-SLS problem: { m } ( (GPS-SLS): min x ai 2 (r + d i ) 2) 2. is tractable (equivalent to GTRS, or least squares) Attainability of the GPS-SLS under rather mild conditions. What about the GPS-LS problem?

47 Illustration of the superiority of CF-LS over CF-SLS (CF-LS) can give pretty good results, but... 5 LS SLS

48 Analysis of the GPS-LS Problem (GPS-LS) : min x { f (x) := } m ( x a i d i r(x)) 2 [ ] 1 m r(x) := ( x a i d i ) m +

49 Existence of an Optimal Solution A related question: what is liminf x f (x)?

50 Existence of an Optimal Solution Theorem A related question: what is liminf x f (x)? ( liminf f (x) = min {(Az + d) T I m 1 ) } x z m 1 m1 T m (Az + d) : z = 1 }{{} f liminf a T 1 a T A :=. 2., 1 m = ones(m,1) a T m f liminf can be efficiently computed via a solution of a GTRS.

51 Auxiliary Lemmata Lemma Let z be an optimal solution of the liminf problem. Then the sequence defined by x k = kz satisfies x k and lim f (x k) = f liminf. k i.e., liminf x f (x) f liminf.

52 Auxiliary Lemmata Lemma Let z be an optimal solution of the liminf problem. Then the sequence defined by x k = kz satisfies x k and lim f (x k) = f liminf. k i.e., liminf x f (x) f liminf. Lemma f (x) A(x)f liminf + C(x) (A(x) 1, C(x) 0). i.e., liminf x f (x) f liminf.

53 Auxiliary Lemmata Lemma Let z be an optimal solution of the liminf problem. Then the sequence defined by x k = kz satisfies x k and lim f (x k) = f liminf. k i.e., liminf x f (x) f liminf. Lemma f (x) A(x)f liminf + C(x) (A(x) 1, C(x) 0). i.e., liminf x f (x) f liminf. liminf x f (x) = f liminf

54 Sufficient Conditions for Attainability [SC1]: there exists x R n such that f ( x) < f liminf Essentially incomputable

55 Sufficient Conditions for Attainability [SC1]: there exists x R n such that f ( x) < f liminf Essentially incomputable [SC2]: f (x sls ) < f liminf, x sls - an optimal solution of (GPS-SLS). Verifiable.

56 Is the sufficient condition likely to be satisfied? m = 6, n = 2. a j and x randomly generated from [ 10, 10] [ 10, 10]. r randomly generated via N(0, 10 2 ) realizations. d j = x a j r + ε j, ε j N(0, σ 2 ). N σ - number of iteration in which [SC2] is satisfied. σ N σ

57 Is the sufficient condition likely to be satisfied? m = 6, n = 2. a j and x randomly generated from [ 10, 10] [ 10, 10]. r randomly generated via N(0, 10 2 ) realizations. d j = x a j r + ε j, ε j N(0, σ 2 ). N σ - number of iteration in which [SC2] is satisfied. σ N σ When σ is not large, [SC2] is satisfied. σ = 10 is a huge standard deviation (the pseudoranges are essentially random).

58 The meaning of f liminf for (CF-SLS) ( f liminf = min {z T A T I m 1 ) } z m 1 m1 T m Az : z = 1

59 The meaning of f liminf for (CF-SLS) ( f liminf = min {z T A T z I m 1 m 1 m1 T m [ = λ min A (I T m 1 m 1 m1 T m ) ) A ] } Az : z = 1

60 The meaning of f liminf for (CF-SLS) ( f liminf = min {z T A T z I m 1 m 1 m1 T m [ = λ min A (I T m 1 m 1 m1 T m ) ) A Question: What is the meaning of this eigenvalue? ] } Az : z = 1

61 The meaning of f liminf for (CF-SLS) ( f liminf = min {z T A T z I m 1 m 1 m1 T m [ = λ min A (I T m 1 m 1 m1 T m ) ) A ] } Az : z = 1 Question: What is the meaning of this eigenvalue? Answer: It is the optimal value of the Orthogonal Regression Problem.

62 The Orthogonal Regression Problem Given a set of points {a 1,..., a m }, find an hyperplane H x,y := { a R n : x T a = y } minimizing the sum of squared Euclidean distances to the set points: { m } f OR = min d(a i, H x,y ) 2 : 0 x R n, y R x,y a 3 a a 2 a a

63 The Orthogonal Regression Problem Given a set of points {a 1,..., a m }, find an hyperplane H x,y := { a R n : x T a = y } minimizing the sum of squared Euclidean distances to the set points: { m } f OR = min d(a i, H x,y ) 2 : 0 x R n, y R x,y Theorem f OR = f liminf a 3 a a 2 a a

64 Circle Fitting versus Orthogonal Regression A sequence of circles with corresponding obj. function value converging to the liminf. 20 k= 1 20 k= k= k=

65 [SC1] for (CF-LS) Revisited [SC1]f < f liminf [SC1] It is better to fit with a circle than with a line

66 A Fixed Point Method for Solving (GPS-LS) First Observation: m m f (x) = ( x a i d i r(x)) 2 = ( x a i d i ) 2 mr(x) 2. P m ( x ai di)2 - obj. function of the source localization problem. mr(x) 2 = m ˆ 1 m P m ( x ai di) 2 - a convex function. +

A Fixed Point Method for Solving (GPS-LS) First Observation: m m f (x) = ( x a i d i r(x)) 2 = ( x a i d i ) 2 mr(x) 2. P m ( x ai di)2 - obj. function of the source localization problem.

67 A Fixed Point Method for Solving (GPS-LS) First Observation: m m f (x) = ( x a i d i r(x)) 2 = ( x a i d i ) 2 mr(x) 2. P m ( x ai di)2 - obj. function of the source localization problem. mr(x) 2 = m ˆ 1 m The Source Localization Problem: Given noisy observations of the distances between the source and the sensors d i x a i, find a good estimate of x. P m ( x ai di) 2 - a convex function. +

68 A Fixed Point Method for Solving (GPS-LS) A generalization of a FP method constructed for the source localization problem (Beck, Teboulle, Chikishev, 2008) A = {a 1, a 2,..., a m }

69 A Fixed Point Method for Solving (GPS-LS) A generalization of a FP method constructed for the source localization problem (Beck, Teboulle, Chikishev, 2008) Optimality condition (x / A): A = {a 1, a 2,..., a m } f (x) = 0.

70 A Fixed Point Method for Solving (GPS-LS) A generalization of a FP method constructed for the source localization problem (Beck, Teboulle, Chikishev, 2008) Optimality condition (x / A): Equivalent to x = 1 m m a i + 1 m A = {a 1, a 2,..., a m } f (x) = 0. m (r(x) + d i ) x a i x a i T (x).

71 A Fixed Point Method for Solving (GPS-LS) A generalization of a FP method constructed for the source localization problem (Beck, Teboulle, Chikishev, 2008) Optimality condition (x / A): Equivalent to x = 1 m m a i + 1 m A = {a 1, a 2,..., a m } f (x) = 0. m (r(x) + d i ) x a i x a i T (x). A fixed point method for solving (GPS-LS) Initialization. Choose x 0 / A. General step. x k+1 = T (x k ), k = 0, 1, 2,...

72 A fixed point method for solving (CF-LS) Initialization. Choose x 0 / A. General Step. [ ] x k+1 = 1 m 1 m x k a i a i + r(x k ), k = 0, 1, 2,... m m x k a i

73 A fixed point method for solving (CF-LS) Initialization. Choose x 0 / A. General Step. [ ] x k+1 = 1 m 1 m x k a i a i + r(x k ), k = 0, 1, 2,... m m x k a i Question I: Can we prove monotonicity/convergence to a stationary point? Question II: Can we avoid the nondifferentiability points A?

74 Convergence Analysis Technique for FP Methods FP method: x k+1 = T (x k ), k = 01, 2,... for solving min{f (x) : x R n }.

75 Convergence Analysis Technique for FP Methods FP method: x k+1 = T (x k ), k = 01, 2,... for solving min{f (x) : x R n }. Analysis Technique: find an auxiliary function h(x, y) for which T (y) = argmin h(x, y) x (x k+1 = argmin h(x, x k )) x

76 Convergence Analysis Technique for FP Methods FP method: for solving x k+1 = T (x k ), k = 01, 2,... min{f (x) : x R n }. Analysis Technique: find an auxiliary function h(x, y) for which T (y) = argmin h(x, y) x f (x) h(x, y) x, y. (x k+1 = argmin h(x, x k )) x

77 Convergence Analysis Technique for FP Methods FP method: for solving x k+1 = T (x k ), k = 01, 2,... min{f (x) : x R n }. Analysis Technique: find an auxiliary function h(x, y) for which T (y) = argmin h(x, y) x f (x) h(x, y) x, y. f (x) = h(x, x) x. (x k+1 = argmin h(x, x k )) x

78 Convergence Analysis Technique for FP Methods FP method: for solving x k+1 = T (x k ), k = 01, 2,... min{f (x) : x R n }. Analysis Technique: find an auxiliary function h(x, y) for which T (y) = argmin h(x, y) x f (x) h(x, y) x, y. f (x) = h(x, x) x. Results: (x k+1 = argmin h(x, x k )) x The FP method is a decreasing scheme (f (x k+1 ) < f (x k )). Any accumulation point is a stationary point. f (x k ) converges to a function value of a stationary point.

79 Convergence Analysis Technique for FP Methods FP method: for solving x k+1 = T (x k ), k = 01, 2,... min{f (x) : x R n }. Analysis Technique: find an auxiliary function h(x, y) for which T (y) = argmin h(x, y) x f (x) h(x, y) x, y. f (x) = h(x, x) x. Results: (x k+1 = argmin h(x, x k )) x The FP method is a decreasing scheme (f (x k+1 ) < f (x k )). Any accumulation point is a stationary point. f (x k ) converges to a function value of a stationary point. Another example: Gradient method. T (y) = y 1 L f (y) h(x, y) = f (y) + f (y), x y + L x y 2 2

80 The Auxiliary Function h(x, y) = m x a i (r(y) + d i ) y a 2 i y a i,

81 Choosing the Initial Point Potential Difficulties: x k A for some k x k.

82 Choosing the Initial Point Potential Difficulties: x k A for some k x k. The solution : Choose x 0 satisfying f (x 0 ) < min{f (a 1 ),..., f (a m ), f liminf }

83 Choosing the Initial Point Potential Difficulties: x k A for some k x k. The solution : Choose x 0 satisfying f (x 0 ) < min{f (a 1 ),..., f (a m ), f liminf } How? f liminf min{f (a 1 ),..., f (a m )} x 0 = x sls

84 Choosing the Initial Point Potential Difficulties: x k A for some k x k. The solution : Choose x 0 satisfying How? f (x 0 ) < min{f (a 1 ),..., f (a m ), f liminf } f liminf min{f (a 1 ),..., f (a m )} x 0 = x sls f liminf > min{f (a 1 ),..., f (a m )}... Pick p argmin{f (a i)}.,...,m Find a descent direction f (a p, d) < 0. Define x 0 = a p + εd. Result: it is always possible to construct x 0 satisfying.

85 Comparing (GPS-LS) and (GPS-SLS) m = 6, n = 2. a j and x randomly generated from [ 10, 10] [ 10, 10] realizations. d j = x a j r + ε j, ε j N(0, σ 2 ). N σ - number of iteration in which [SC2] is satisfied. σ rel. er. SLS rel. er. LS I σ

86 Comparing (GPS-LS) and (GPS-SLS) m = 6, n = 2. a j and x randomly generated from [ 10, 10] [ 10, 10] realizations. d j = x a j r + ε j, ε j N(0, σ 2 ). N σ - number of iteration in which [SC2] is satisfied. σ rel. er. SLS rel. er. LS I σ I σ - no. of runs in which the LS solution is better than the SLS solution ( ) rel. er. SLS (LS) - average of xsls xtrue xls x true x true x true.

87 The End Thank you!

Additional Exercises for Introduction to Nonlinear Optimization Amir Beck March 16, 2017

Additional Exercises for Introduction to Nonlinear Optimization Amir Beck March 16, 2017 Chapter 1 - Mathematical Preliminaries 1.1 Let S R n. (a) Suppose that T is an open set satisfying T S. Prove that