Another face of DIRECT

Size: px

Start display at page:

Download "Another face of DIRECT"

Collin Hampton
5 years ago
Views:

1 Aother ae o DIEC Lakhdar Chiter Departmet o Mathematis, Seti Uiversity, Seti 19000, Algeria address: hiterl@yahoo.r Abstrat It is show that, otrary to a laim o [D. E. Fikel, C.. Kelley, Additive salig ad the DIEC algorithm, J. Glob. Optim. 36 ( ], it is possible to divide the smallest hyperube whih otais the low utio value by osiderig hyperretagles whose poits are loated o the diagoal o the eter poit o this hyperube usig a divisio proedure whih iluees the slope to be below the the threshold mi - ε mi ad thus redues the iluee o the parameter ε. Keywords: DIEC ; Global optimizatio; Geometrial iterpretatio 1. Itrodutio he DIEC (DIvidig ECagles algorithm o Joes et al. [] is a determiisti samplig method desiged or boud ostraied o-smooth problems. here are two mai ompoets i DIEC: oe is its strategy o partitioig the searh domai, the other is the idetiiatio o potetially optimal hyperretagles, i.e., havig potetial to otai good solutios. he relative potetial o eah partitio is haraterized by two attributes: the value o the objetive utio at the eter o the partitio, ad its size. he ormer provides a measure o the potetial o the partitio with respet to loal searh, i.e., partitios with good utio values at their eter are more desirable tha those with worse utio values. he latter provides a measure o the partitio s potetial with respet to global searh, that is, larger partitios otai more uexplored territory, ad thereore provide a greater opportuity or urther exploratio. A parameter is used to otrol the balae betwee loal ad global searh ad protet the algorithm agaist exessive emphasis o loal searh. he eets o this parameter ad its iluee o the rapidity o the overgee were studied i a reet paper by Fikel ad Kelley [1]. A algorithm that plaes too muh emphasis o loal searh will easily be trapped ear a loal optimum. Coversely, a algorithm that speds too muh time perormig global searh will overge very slowly. hereore, a tehique that does ot deped o suh parameter would be desirable. his paper is oered with suh tehiques. As reported by may authors, the priipal disadvatage o DIEC is its slow loal overgee, i.e, whe the size o hyperretagles beomes too small. Ater may iteratios o DIEC the smallest hyperretagle otaiig mi (the urret best utio value is always a hyperube, this hyperube will ot be adidate or seletio beause o the parameter ε. Fikel ad Kelley [1], i their paper (see thm. 3.1., show that the hyperube, whih otais the urret miimal poit will ot beome potetially optimal util all larger hyperretagles havig their eters o the steil are the same size as o this hyperube, i.e., ater all these hyperretagles have bee divided.

2 he aim o this paper is to show that the result o theorem 3.1 i [1] is ot eessary relevat. he at that the hyperube with the low utio value is rejeted rom seletio beause o the iluee o hyperretagles whose poits (eters are o the steil, it does ot ollow that this will happe with hyperretagles whose poits (quarters are o the diagoal. his should ot be eessary the ase as we shall see i set. 4. We propose a modiiatio to DIEC. he modiied method produes more hyperretagles tha the origial DIEC by evaluatig the objetive utio at the quarters o eah hyperretagle, thus miimizig the umber o evaluatios. Eah hyperretagle i the divisio will have, ater divisio, two poits. Eah poit a be see as a quarter (1/4 or (3/4 i the other ae aordig to eah diretios. Usig this proedure, we a disard all potetially optimal hyperretagles o the steil whih prevet the smallest hyperube or beig optimal, by osiderig the poits o the diagoal. We a seek the values o that allows us to have a suiiet derease o the slope to the let o the potetially optimal hyperretagle ad thus ireases the slope to the right o the smallest hyperube. Modiiatios usig spae partitioig tehique have bee ivestigated by may authors, see or example, tree-diret [5]. his paper is orgaized as ollows: I the ext setio, we give a short desriptio o DIEC. I Setio 3, we give the geometrial iterpretatio o oditio 5 o theorem 3.1 o [1]. his allows us to desribe our modiiatio to DIEC i setio 4, ad give a weaker oditio whih prevets the smallest hyperube or ot beig optimal. he we olude i setio 5.. DIEC he DIEC algorithm begis by salig the desig domai, Ω, to a -dimesioal uit hyperube. his has o eets o the optimizatio proess. DIEC iitiates its searh by evaluatig the objetive utio at the eter poit o, = (1/,..., 1/. is idetiied as the irst potetially optimal hyperretagle. he DIEC algorithm begis the searh proess by evaluatig the utio i all diretios at the poits ± δe i whih are determied as equidistat to the eter. Where δ is the oe third o the distae o the hyperube, ad e i is the i th uit vetor. he DIEC moves to the ext phase o the iteratio, ad divides the irst potetially optimal hyperretagle. he divisio proedure is doe by trisetig i all diretios. he trisetio is based o the diretios with the smallest utio value. his is the irst iteratio o DIEC. he seod phase o the algorithm is the seletio o potetially optimal hyperretagles. A deiitio or this is is give below. Samplig o the maximum legth diretios prevets boxes rom beomig overly skewed ad trisetig i the diretio o the best utio value allows the largest retagles to otai the best utio value. his strategy ireases the attrativeess o searhig ear poits with good utio values. More details about DIEC a be oud i []. Deiitio.1. Assumig that the uit hyperube with eter i is divided ito m hyperretagles, a hyperretagle j is said to be potetially optimal i there exists rate-o-hage ostat K suh that ( j Kd j ( i Kd i, or i = 1,..., m (.1

3 ( Kd ε (. j j mi mi Where mi is the best utio value oud up to ow, d i is the distae rom the eter poit to the verties, ad the parameter ε is used here to protet the algorithm agaist exessive loal bias i the searh. he set o potetially optimal hyperretagles are those hyperretagles deiig the bottom o the ovex hull o a satter plot o hyperretagle diameter versus ( i or all retagle eters i, see Fig.1. I this graph, the irst equatio (.1, ores the seletio o the retagles i the lower right ovex hull o dots. Coditio (.1 a be iterpreted by the slopes o the liear urves represeted to the right ad to the let o the poit P(d i, ( i. I the slopes o the urves passig through P ad the poits o the right o this oe are all greater tha those passig by P ad the poits o the let o this oe, the there exists some K >0 veriyig (.1. he oditio (. ores more the hoie o boxes i terms o size. I at, the hyperretagle i will be seleted oly i the slopes o urves o the right o P are greater tha the lie passig by P ad mi. his allows ot to selet very small boxes ad so to stop the overgee earlier. he seletio preseted here allows to explore at the same time boxes with importat sizes to realize a global searh ad boxes with small sizes to arry out a loal searh. he parameter iluees the slope o the lie passig by P ad mi. More this slope is weak (ε = 0, more hyperretagle with small size are seleted ad thus we do a loal searh. I ε is lose to 1, the slope o this urve is more strog ad oly ew hyperretagles o small size are seleted. We have the a global searh. 3. Geometrial iterpretatio Fig1. Iterpretatio o deiitio.1. I this setio we desribe how the smallest hyperube with the lowest utio value, ( = mi, a be disarded rom beig optimal, i.e., i the ses that it does ot satisy oditio (., the oditio whih uses the parameter ε. We will urther give a geometrial iterpretatio

4 o this rom a ie theorem due to Fikel et ad Kelley [1]. I this paper we are ot oered with this parameter. I the Fig., we a see that the slopes are stroger to the right, this is the global part o the algorithm. As the algorithm otiues, the searh will be loal, ad the size o the hyperretagles beomes too small ad thus the slopes are too weaker. his prevets to ot seletig hyperube with small utio value. I the Fig., the square dot alters the lower ovex hull, ad the small hyperretagle, whih otais the low utio value is ot potetially optimal. he lie goig through the poits (, ( ad (, ( aot be below the quatity mi - ε mi. his is due to the larger hyperretagle to the right havig a omparable value o at its eter. For best uderstadig, the ollowig theorem (see [1] explais how a hyperretagle otaiig mi does ot satisy oditio (.. heorem 3.1. (see [1] Let : be a Lipshitz otiuous utio with Lipshitz ostat K. Let S be the set o hyperretagles reated by DIEC, ad let be a hyperube with a eter ad side legth 3 -l. Suppose that (i, or all S (i.e. is i the set o smallest hyperubes. (ii ( = mi 0 (i.e. ( is the low utio value oud. I ( + ε ( < 8 (3.1 K Fig.. he geometrial iterpretatio o oditios (. ad (3.1. the will ot be potetially optimal util all hyperretagles i the eighborhood o, i.e., all hyperretagles whose eters are o the steil ±3 -l e i or i = 1,...,N are the same size as. emark 3.1. Coditio (3.1 i the above theorem a be iterpreted by the the ollowig iequality

5 ε ( > ( ( (3. Where is the size o a the smallest hyperretagle, (see [1] or details. I at, i oditio (. i the deiitio.1, is ot satisied, i.e., ( K > ε mi mi he ε ( K < But, K = ( (, ad K K + 8 However, the oditio i i the above theorem shows that i oditio (3. is satisied, the the hyperube will ot be potetially optimal. Geometrially, oditio (3. is represeted i Fig., by the taget o the agle φ whih is greater tha the taget o the agle ψ, where ε ( ta φ =, ad ta ψ = ( ( I their paper, Fikel ad Kelley. [1], have hose to redue the iluee o the parameter ε. heir modiiatio to DIEC relates to a update to the deiitio o potetially optimal hyperretagles. Modiiatios related to the parameter ε a also be oud i [3] or small values, i.e., ε 0. I her thesis, user [6] suggested to reate a optimizatio proess usig DIEC, but rom a ertai size o a hyperretagle, we add a searh method by deset to overge rapidly. Our modiiatio is related to the potetially optimal hyperretagle, whih iluees the slope ad the prevetig hyperube or beig optimal, see Fig. We seek or values o that allows us to have a suiiet derease o the slope to the let o hyperretagle ad thus ireases the slope to the right o hyperube. 4. New iterpretatio o DIEC his setio shortly desribes some hages to DIEC. We suggest to hage the way a hyperube is divided. Istead o trisetig a hyperube aordig to the strategy o the lowest utio values, we suggest to use a divisio as desribed below. A hyperretagle is oly biseted oe alog its logest side. his meas that, this ireases the umber o hyperretagles, thereore the searh is doe irstly more global. Oe all hyperretagles have bee divided, the searh beomes more loal. his strategy does ot plaes the lowest utio values i the largest hyperretagles. For a better uderstadig we start with a iterval ad the

6 desribe the more geeral ase o a hyperretagle. DIEC a be see as ollows: risetig a iterval is equivalet to divide it i two iequal subitervals, suh that oe is hal o the other. his is doe i the ratio 1/3 or /3 as illustred i Fig.3. he we divide the large iterval i two equal subitervals. I two dimesios, the orrespodig divisio is show i Fig. 4. emark that there is o dieree with the divisio proedure (a or (b, sie the value o at this eah poit (exept the eter is the same or the two shaded domais, as show i Fig. 4, beause we do ot eed to leave the best utio value i a largest spae as i DIEC, ad the size whih is represeted by the logest distae rom the evaluatio poit to the verties is the same. Eah poit, exept the oe o the eter, belogs to N hyperretagles i all diretios, where N is the dimesio o the domai. he size o a hyperretagle is represeted by the logest distae rom the evaluatio poit whih is loated i the quarters o the retagle to oe o its orers. his measure a be iterpreted as the hal o a hyperretagle or the DIEC, whih was alled i [-4] by the l diameter. For more details about the size we reer to[4]. he poits sampled are equidistat rom the eter. he distae rom the eter to eah poit i the steil is represeted by -l, where 3. -l is the side leght o a hyperube, as show i Fig. 5. he utio is evaluated at ± -l e i, where e i is the ith uit vetor. I the Fig. 6 we show a example o the irst three iteratios. Fig.3. Divisio o a iterval. I (, it does ot matter whih divisio we hoose (a or (b, sie the value o at this poit is the same.

7 Fig.4. divisio o a square. As or oe dimesio, we a hoose the divisio (a or (b. Fig.5. Size o a hyperretagle. he distae rom the sampled poit to the vertex a be see as the diameter o the small irosribed retagle.

8 Fig. 6. A example o the irst three iteratios. he divisio o a three dimesioal retagle is represeted i Fig. 7. the oly dieree i the above examples, is that this divisio takes ito aout the at that the lower utio value is i a larger spae, oly i the irst stage o the divisio. Fig. 7. Dividig a three dimesioal retagle I what ollows a similar result as i theorem 3.1 o [1], with a weaker oditio. We adopt the otatios o theorem 3.1, ad we a hoose either -l or 3 -l as the size o the smallest hyperube. he ollowig theorem shows that i we are i the oditios o theorem 3.1, i.e., a hyperube aot be potetially optimal, it will ot be eessary the ase or hyperretagles havig their

9 poits o the diagoal. But i oditio (4.1 is satisied the oditio (3.1 holds. his estabilishes the ollowig theorem. heorem 4.1. Let : be a Lipshitz otiuous utio with Lipshitz ostat K, Let S be the set o hyperretagles reated by DIEC, ad let be a hyperube with a eter ad side legth -l. Suppose that (i ad ( o theorem 3.1. are satisied. I ( + ε ( < 8 (4.1 K the will ot be potetially optimal util all hyperretagles whose poits are o the diagoal l ± are the same size as. i.e., will either be optimal or hyperretagles o the steil ± l e, or or those o the diagoal. i emark 4.1. heorem 3.1 is still valid or our ase, sie the measure o a hyperretagle is hal o the diameter o the a hyperretagle as show i Fig. 5. Note that we a use either 3 -l or -l as the smallest side legth, ad the same olusio holds, or the simple raiso that the terms 3 -l or -l will be simpliied as see i theorem 3.1 o [1]. A hyperretagle S will have - 1 sides o legth (3 -l / ad oe side o legth 3 - l.(3/, i.e., = l 3 ( l 3 l 3 = + 8 Fig.8. Poits represeted o the steil (a, ad i (b o the diagoal. Ad i we use -l l1 l1 as a size we will have = ( 1 ( + ( 3. = ( + 8 Proo. he irst airmatio is immediate sie l ε ( K ( ε ( + 8 ( + 8, or 1. K

10 It is easy to remark that i two dimesios, the poits o the diagoal are ± 3 -l (e 1 ± e, ad the distae rom to these poits is l. For a three dimesioal ube, this distae is l 3, l ad or a hyperube we get. We adopt the same proo as i [1]. I all hyperretagles i Fig.8 satisy oditio 3.1, we a use hyperretagles havig their poits o the diagoal, sie the size is the same i (a or (b. We get the ollowig orollary. Corollary 4.. Let ad be as i theorem 3.1. Suppose that oditios (i ad (ii o theorem 3.1 holds. I K ( ( + 8 ( + 8 ε ( ε K (4. the will be potetially optimal or all hyperretagles whose poits are o the diagoal l ±. By usig the poits o the diagoals or a potetially optimal hyperretagle, we get a orretio o the lower oex hull as see i Fig. 9. Proo. he right had side o iequality meas that will ot be potetially optimal or hyperretagles o the steil. While the let had side meas that, will be potetially optimal, i.e., satisy oditio (. o deiitio.1. For hyperube to be potetially optimal there must exist K suh that (.1 ad (. hold. From oditio (.1 we get K ( ( we must hoose K = max ( ( = ( (

11 Fig.9. A orretio o the lower ovex hull. Hyperretagle whih satisy oditio (3.1 o theorem 3.1 is disarded ad replaed by aother hyperretagle or whih the value o is muh greater. he let had side o iequality (4. is equivalet to ; 8 ( K l + ε (4.3 ad ( l + 8 he Lipshitz otiuity o is equivalet to. ( ( K l hus ( K K K l l + = + = 8 8 ( ( From the right had side o (4. ad iequality (4.3, we get ( K l ε, i.e., mi mi ( K ε.

12 emark 4.. he set o poits ( suh that the hyperube must be potetially optimal are the poits ((, whih are betwee the lies (D ad (D as see i Fig. 10. Let y = ax + b, be the equatio o (D. he ε ( y = x + ( ε ( I ((, (D, we get ( ( ε ( =, the the poits or whih the hyperube must be potetially optimal are suh that ( ( ( ( ε (. he right had side o the above iequatio is to satisy oditio (. o deiitio.1, while the let iequality is due to the ovexity o the lower ovex hull. Fig. 10. he poits or whih the hyperube must be potetially are those poits suh he right had side o the above iequatio is to satisy oditio (. i deiitio.1, ad the seod iequality to the let, beause o the ovexity o the lower ovex hull. 5. Colusio I this paper we have preseted a modiiatio i the divisio proedure or the DIEC algorithm. he hyperretagles reated a have may aes depedig o the dimesio. For

13 eah potetially optimal hyperretagle the size is measured as the logest distae rom the evaluatio poits, whih are loated o the quarters, to its verties. DIEC uses a parameter whih iluees the smallest hyperretagle otaiig mi (the urret best utio value or ot beig potetially optimal, ad thus may iluees the overgee o the algorithm. he smallest hyperretagle is always a hyperube, whih depeds o the hyperretagles whose eters are o the steil o the eter poit o this hyperube. We a disard the potetially optimal hyperretagle, whih iluees the slope ad the prevetig the smallest hyperube or beig optimal, by osiderig hyperretagles havig their poits o the diagoal. he it is possible to id a potetially hyperretagle or whih the smallest hyperube will be potetially optimal. eerees [1] D. E. Fikel, C.. Kelley, Additive salig ad the DIEC algorithm, J. Glob. Optim. (006 36: , Spriger, 006. [] D.. Joes, C.D. Perttue, B.E. Stukma, Lipshitzia Optimizatio without Lipshitz Costat, Joural o Oprtimizatio heory ad Appliatios, Vol. 79, No. 1, Otober [3] J. M. Gablosky, C.. Kelley, : A loally-biased orm o the DIEC algorithm. J. Global. Optim. 1, 7-37, 001. [4] J.M. Gablosky, Modiiatios o the DIEC Algorithm, Ph.D. hesis, North Carolia State Uiversity, aleigh, North Carolia, 001. [5]. E. Haskell, C. A. Jakso, ree-diret: A Eiiet Global Optimizatio Algorithm, Pro. Iteratioal ICSC Symposium o Egieerig o Itelliget Systems, Uiversity o La Lagua, eerie, Spai, February 11-13, [6] K. J. user, Méthodologies pour la plaiiatio de réseaux loaux sas-il, PhD hesis, Frae: INSA de Lyo, 7 otobre 005, 30p. (i Freh, dowloadable rom :

Observer Design with Reduced Measurement Information

Observer Design with Reduced Measurement Information Observer Desig with Redued Measuremet Iformatio I pratie all the states aot be measured so that SVF aot be used Istead oly a redued set of measuremets give by y = x + Du p is available where y( R We assume