A dual control problem and application to marketing

Transcription

1 European Journal of Operational Research 13 (21) 99±11 Theory Methodology A dual control problem application to marketing Gila E. Fruchter * Faculty of Industrial Engineering Management, Israel Institute of Technology, Technion City, Haifa 32, Israel Received 1 February 1999; accepted 12 November 1999 Abstract The general approach to adaptive dual control is to formulate an optimal stochastic control problem. However, for such an approach only mathematical representations of the solution are available which allow little insight into the structure of the optimal controller. Here, an alternative deterministic approach is presented based upon determining a control in which a disturbance attenuation function remains bounded for all allowable (L 2 functions) disturbances. The disturbance attenuation function is composed of the ratio of an L 2 function of the desired outputs over an L 2 function of the disturbance inputs. This disturbance attenuation problem is converted to a di erential game. For this game, the optimal control law, in a closed-form, is obtained by performing a minmax operation with respect to a quadratic cost function subjected to a bilinear system. The resulting controller is time-varying depends nonlinearly on the state the parameter estimates vector, on an associated Riccati-type matrix. We provide insights into the structure of the resulting dual controller illustrate the method by two examples. One of the examples is an application to marketing, to set promotional spending of a company, considering that the e ect of promotional e ort on sales is unknown. Ó 21 Elsevier Science B.V. All rights reserved. Keywords: Adaptive control; Dual control; Estimation; Marketing; Promotion 1. Introduction Considering a linear system in the presence of disturbance parameter uncertainties, we present a game-theoretic approach to a worst case design problem. Formulating the problem as a deterministic game, the best control for the worst * Tel.: ; fax: address: iergf2@techunix.technion.ac.il (G.E. Fruchter). case initial states measurement disturbances is found. The objective is to use a game-theoretic approach to design an optimal adaptive controller. Such a controller can be regarded as composed of two parts: a nonlinear estimator a feedback regulator. Presently, as summarized in Astrom Wittenmark (1989), Whittle (1991) Basar Bernhard (1991), the method of nding adaptive controllers is based upon the certainty equivalence principle. In contrast to this kind of adaptive controller, we want to add an active learning feature to the controller. In this case, the optimal /1/$ - see front matter Ó 21 Elsevier Science B.V. All rights reserved. PII: S ( ) 25-4

2 1 G.E. Fruchter / European Journal of Operational Research 13 (21) 99±11 controller will be a dual controller. It is a compromise between maintaining optimal control small estimation errors. Note that such a controller, with the property of learning control, was rst proposed by Feldbaum (1965). A general approach for this problem is presented, see Astrom Wittenmark (1989), by formulating it as a stochastic control problem. In stochastic optimal control problems, in general, see for example the review paper of Neck (1984), it will not be possible to hle the dual nature of the problem, i.e., to obtain jointly optimal statistical identi cation control of the system. Bryson Ho (1975) made an attempt to apply a deterministic game-theoretic formulation to a type of dual control problem. However, it was shown that even a very simple problem exhibits very complex features. While it is natural to model the unknowns as rom processes nd the control to minimize the expected value of the performance index, so little is known about the statistics that an adequate stochastic formulation is not available. In this paper, an alternative approach is taken by formulating a deterministic disturbance attenuation problem. In the disturbance attenuation problem, a controller is sought such that disturbance attenuation is bounded for all L 2 disturbance inputs. The disturbance attenuation function is an input±output representation constructed as the ratio of an L 2 function of the performance outputs over an L 2 function of the input disturbances, cf. Basar Bernhard (1991). This disturbance attenuation problem is converted into a di erential game the quadratic cost criterion is constructed to be minimized by the control maximized by the input disturbances. A game-theoretic approach to the linear quadratic regulator problem has been the subject of intense research e orts, see Whittle (1981) Basar Bernhard (1991). The main purpose of this paper is to formulate a game for a special type of dual control problem gain insights into some features that do not arise for linear systems. Adopting the game-theoretic formulation, we nd a method to solve the problem analytically, in a closed form. In Section 2, we consider a linear system with unknown initial conditions, measurement disturbances without process disturbances, but with uncertainty, in the coe cient of the control. The dynamics of the uncertainty are described by a linear di erential equation. To solve for a dual controller, i.e., a controller that estimates the uncertainty regulates the system, the linear system is transformed into a bilinear system, the new state is composed of the original state the uncertain parameter. Formulating the game problem, the disturbances initial states are the adversaries, we are maximizing the cost function with respect to initial values the measurement disturbances minimizing with respect to the control. The order of the extremization with respect to the adversaries is important to determine controller strategies that are explicit functions of the measurement sequence. This is done by rst maximizing the game cost criterion with respect to the process disturbance, in our case, the unknown condition n. The resulting controller is time-varying depends nonlinearly on the state parameter estimate vectors their associated Riccati matrix. An insight into the structure of the dual controller is given in Section 5. The dual controller, developed in this paper, can be applied to nd marketing strategies. In Section 6, we illustrate such an application to set promotional spending of a company, considering that the e ect of promotional e ort on sales is unknown. Several marketing researchers dealt with measuring the e ect of promotion (as advertising) on sales, among them, Clarke (1976), Banks (1965), Rao (197), Pekelman Tse (198), Little (1979), Eastlack Rao (1986), Mahajan Muller (1986), Winer Moore (1989). However, the method proposed in this paper o ers an adaptive control scheme that has the ability to simultaneously estimate the e ects on line when new data appear to regulate the promotional setting by diminishing the market noise unknowns e ects. This is especially useful in new products, when data on sales rates are not available on how to use econometric methods to measure the promotional spending e ect on the sales rate. The idea of applying adaptive control theory

3 G.E. Fruchter / European Journal of Operational Research 13 (21) 99±11 11 to managerial problems has been used by many other researchers, among them, Deissenberg Stoppler (1983). 2. Problem formulation Consider the continuous dynamical system with parameter uncertainty in the coe cient of the control given by _x t ˆA 1 t x t k t u t ; _k t ˆA 2 t k t : 1a 1b Here x; k 2 R n, x is the state, k is a vector of unknown parameters in the control coe cient, modeled by Eq. (1b), A 1 ; A 2 2 R nn are given time-varying matrices, u 2 R is the input control, respectively. A linear measurement of the state, y 2 R m, containing an additive disturbance, v 2 R m, is also assumed: y t ˆH t x t v t ; H 2 R mn is a time-varying matrix. De ne the measurement history up to t as Y t ˆfy s : 6 s 6 tg: 1c The admissible control is restricted to be a function of only the measurement history Y t. Let n T ˆ x T k T : Then (1a)±(1c) can be described by the following continuous bilinear system: _n ˆ An Bun; y ˆ En v; A ˆ A1 A 2 ; B ˆ I ; 2a 2b I is the identity matrix, E ˆ H Š. Now, consider the following performance index, motivated by the disturbance attenuation problem, 1 see Basar Bernhard (1991) the `stress' performance index of Whittle (199, 1991): J ˆ J u; v; n ˆ 1 2 kn T k2 Q T 1 h kn ^n k 2 P 1 Z T u 2 1 h kvk2 V dt : 3 1 h Here, ^nt ˆ ^xt ; ^k i T, h is a negative constant, P V ˆ V t are positive de nite symmetric matrices Q T is a constant nonnegative de nite symmetric matrix, of compatible dimensions, respectively. Our objective is to nd a control which is a function of the measurement history up to t; u Y t 2L 2 ; T Š, which will be the best adaptive control for the worst case disturbance v 2 L 2 ; T Š worst case initial condition n. Such a control will minimize the payo functional for worst case disturbance initial state by estimating the uncertainty. As will be shown as expected, since the system is bilinear, control estimation cannot be designed separately. The estimation is a ected by the control, which can perform the dual roles of regulation learning (the uncertainty k). Therefore, the adaptive controller will be a dual controller, it a ects the acquisition of information as well as the regulation of the process. Considering this problem as a deterministic game, the optimal dual controller is obtained by performing the following minmax operations: min u max max J; v n 1 The disturbance attenuation problem is to nd the admissible control u(y t ) such that the attenuation function, de ned as the ratio of quadratic functions of the output performance measures (represented here by the states control) to the quadratic functions of the input disturbances including the initial state, is bounded as kn T k 2 R T QT u2 dt kn ^n k 2 P R T 1 kvk2 V dt < 1 h 1 8v 2 L 2 ; T Š; n 2R 2n ; h < :

4 12 G.E. Fruchter / European Journal of Operational Research 13 (21) 99±11 subjected to Eqs. (2a) (2b). In this game, the control u wants to minimize exactly what the adversaries n v want to maximize. The maximization operation belongs to the estimation process the minimization operation gives us the optimal desired dual control law. The desired solution is obtained in two steps: (i) We nd the optimal open-loop strategy. For this step we do not need to separate the maximization operations. (ii) We use the open-loop strategy for nding the desired closed-loop strategy. In this step we use the results obtained by separating the maximization operations. Remark 1. The separation the order of the maximization problem are important for our nal goal to nd a closed-loop strategy that is an explicit function of the measurement. If the cost criterion (3) is maximized with respect to n, assuming u y are given, then a state reconstruction scheme is obtained. Given this state estimate along with u y, the measurement disturbance can be computed. This state reconstruction scheme is found in Section 3.1. In Section 3.4, the maximization with respect to the residual of the state reconstruction is performed resulting in a nonlinear optimal control in u. Note that the cost criterion for the continuous deterministic game reminds us of the criterion of the Linear-Exponential-Gaussian stochastic control problem cf. Whittle (1981) Besoussan Schuppen (1985). 3. Estimation of the uncertainty 3.1. Maximization with respect to the initial condition n " # _n ˆ _k A Bu E T V 1 E A Bu T ; E T V 1 y n k n ˆ^n P k ; k T ˆhQ T n T : 4 k represents the Lagrange multiplier (adjoint variable) y is de ned in (2b). Since P is positive de nite h <, the necessary condition (4), for the initial condition, is also su cient it is a local maximum for the cost function J. For solving problem (4), we use the sweep method as in Bryson Ho (1975) Derivation of estimator structure Let n t ˆ^n t P t k t ; t 2 ; T Š; 5a ^n P are to be determined. Di erentiating (5a) considering (4) again (5a), we obtain _n ˆ A Bu ^n A Bu Pk ˆ _^n _Pk P _ k ˆ _^n _Pk P E T V 1 E ^n Pk A Bu T k E T V 1 yš: Rearranging this equation, we obtain _^n A Bu ^n PE T V 1 E ^n y h ˆ _P PE T V 1 EP P A Bu T i A Bu P k: 5b In conclusion, for any t 2 ; T Š, an estimator can be chosen so that _^n ˆ A Bu ^n PE T V 1^v; ^n ˆ^n ; 6 Considering the stard variational problem for the cost J de ned in (3) subjected to the dynamical system (2a) (2b), with xed arbitrary strategies v u, the following two-point boundary value problem can be obtained: ^v ˆ y E ^n P satis es _P ˆ A Bu P P A Bu T PE T V 1 EP; P ˆP : 7

5 G.E. Fruchter / European Journal of Operational Research 13 (21) 99±11 13 The estimator, de ned by (6) (7), has a role similar to the Kalman lter Cost criterion as a function of lter variables In the following, we want to rewrite J in terms of the lter variables, ^n ^v: Considering the rst-order necessary condition (4), we obtain J ˆ 1 2 kn T k2 Q T 1 h kk k2 P Z T u 2 1 h kvk2 V dt : 1 Now consider the zero sum ˆ 1 Z 2h kt Pkj T T d dt kt Pk dt : Adding this zero sum to the above J, considering Eqs. (2b), (4)±(5b) (7), we obtain, after some manipulations, J ˆ 1 2 kn T k2 Q T 1 h kk T k2 P T Z T u 2 1 h k^vk2 V dt 1 ˆ 1 2 kn T k2 Q T I hp T Q T ˆ 1 Z T 2 k^n T k 2 S T S T ˆ Q T I hp T Q T Š 1 : u 2 Z T u 2 1 h k^vk2 V dt 1 1 h k^vk2 V dt ; 1 8a 8b Note that we assume that I hp T Q T is a nonsingular matrix Maximization with respect to ^v Repeating now the variational method to the cost function (8a) (8b) subjected to Eq. (6), for xed arbitrary u, the following two-point boundary value problem is obtained: " # _^n ˆ A Bu " # hpet V 1 EP ^n _^k A Bu T ; ^k ^n ˆ^n ; 9 ^k T ˆS T ^n T ; ^k is the Lagrange multiplier. Note that here the Hamiltonian is H ˆ 1 2 u2 1 h k^vk2 V 1 ^k h T A Bu ^n PE T V 1^v i : Since V is positive de nite h < ; o 2 H=o^v 2 ˆ hv 1 is negative de nite therefore the necessary condition ^v ˆ hep ^k, is also su cient, it is a local maximum for J. For solving (9), we use again the sweep method de ne a new variable ^n ˆ n hp ^k; t 2 ; T Š: 1 For nding n, we di erentiate (1) consider (9) (1), obtain _^n ˆ A Bu ^n hpe T V 1 EP ^k ˆ A Bu n h A BuŠP P E T V 1 EPŠ^k ˆ _ n h _P ^k P _^k ˆ_ n h _P P A Bu T Š^k: Considering this equation, we have _n A Bu n h ˆ h _P P A Bu T A Bu P PE T V 1 EPi^k: 11 Considering (7) (11) we obtain that the new variable has to satisfy _n A Bu n; n ˆ^n hp ^k : 12 The cost function in (8a) (8b) in terms of the new variable will become

6 14 G.E. Fruchter / European Journal of Operational Research 13 (21) 99±11 J ˆ 1 n T 2 2 Q T 1 n h ^n 2 P 1 Z T u 2 dt : Determination of the optimal dual control law 4.1. Minimization with respect to u For performing the last step of the variational problem, we have to consider the cost function in (13) subjected to Eq. (12). In this case, the following necessary conditions are obtained: _g ˆ A Bu T g; g T ˆQ T n T ; 14 u ˆ g T B n; 15 Remark 2. Note that the special de nition in (1), of choosing hp for the coe cient of ^k, is crucial essential, is the key in solving the problem in this paper. Due to this de nition, rst, we reduce the complexity of the computation in the minimization process (since we do not have new constraint equations, i.e., an additional Riccati equation, as usually by the stard method, see condition (11)). Secondly, the identity (17) is a result of this de nition The open-loop optimal strategy The open-loop optimal control law is obtained by Eq. (15), g; n is a solution of the twopoint boundary problem obtained from (14) (18) by substituting (15) for u, i.e., g represents the Lagrange multiplier (adjoint variable). The Hamiltonian in this case is H ˆ 1 2 u2 g T A Bu n; o 2 H=ou 2 ˆ 1 >, therefore the necessary condition (15) is also su cient, it is a local minimum for the cost function J. Another way to write the identity (8b) can be S T ˆ Q T I hp T S T Š: Considering this identity the boundary condition of the second equation in (9), (1) at t ˆ T, we obtain ^k T ˆS T ^n T ˆQT I hp T S T Š^n T ˆ Q T ^n T hp T ^k T Š ˆ Q T n T : 16 Comparing now the second equation in (9), the boundary condition is in the form (16), with the condition (14), we conclude with g ˆ ^k: Considering (17) (12) will become 17 _n ˆ A Bu n; n ˆ^n hp g : 18 _g ˆ A Bg T Bn T g; g T ˆQ T T ; _n ˆ A Bg T Bn n; n ˆ^n hp g : 19 An analytical solution to this problem is illustrated by an example in Section 5. For more complicated cases, one can use a numerical method, like the shooting method, cf. Ascher et al. (1988) Lambert (1973). Denote the solution in the following form: u ol ˆ u ^n ; P ; T ; t ; t 2 ; T Š; 2 the subscript denotes open loop. As mentioned in the next section, we illustrate how to nd u explicitly. Therefore, in the sequel we can regard u as a known function. Without any further information (at this stage we do not need Eqs. (6) (7)), Eq. (2) is the best possible control. In the following, we will show how this open-loop strategy, together with the estimator Eqs. (6) (7), can be used for nding the feedback solution which will depend on the actual measurements The closed-loop dual controller Consider Eq. (2), u is known calculated `o line' by (15) (19), as presented

7 G.E. Fruchter / European Journal of Operational Research 13 (21) 99±11 15 above, the initial conditions are ^n t P(t) rather than ^n P. Also, the nal time is the time to go, T go ˆ T t; 21 rather than T. Then the optimal feedback controller will be u t ˆ uol; t ˆ ; 22 u ^n t ; P t ; T go ; t ; t 6ˆ : The real time initial conditions ^n t P(t) are estimated from Eqs. (6) (7). Note that the initial measurement y() is found by using the input u ˆu ol, known in advance. Next we illustrate by an example how one can nd the control law (22). 5. Interpretation of the results In Fig. 1, we nd a schematic interpretation of the results presented here. The control u is used both for estimating ^n ˆ ^x T ^kt Š regulating the system Eqs. (1a) (1b). For this reason, u is called the dual controller. Starting at t ˆ ; u is applied to the system Eqs. (1a) (1b) for nding the current measurement y( ), which together with the input u ˆ u ol (calculated in advance Fig. 1. Description of the dual controller structure. on `o line') is applied to the system (6) (7) for estimating ^n t, P(t), in real time, by the `Estimation Box'. The desired time-varying feedback control law u (t) follows by using the new data of `initial values at instant t', ^n t, P(t), instead of the initial values ^n, P, the time to go T go, instead of the nal time T, in the `Regulation Box'. 6. A marketing application The particular case, (1a)±(1c) is a scalar system, furthermore A 1 ˆ A 2 ˆ ; x t is scalar k is constant but unknown, can be useful to conduct a companyõs marketing operations such as promotional spending. Company sales are functions of promotional spending, but the relation changes with time. The state x(t) denotes the sales rate of a company at time t, u(t) the promotional e ort at time t. It is assumed that promotional e ort u(t) represents the square root of the companyõs promotional spending; this is a common assumption in the literature of advertising models, cf., Erickson (1992) Fruchter Kalish (1997). In most of the cases, especially for new products, the e ect of promotional e ort on sales is unknown. We represent it by k. Eq. (1a) describes the dynamic relationship between the change in sales rate the promotional e ort its e ect on sales. Eq. (1c) takes into account the market noises, v, in measuring the sales rate at time t, as y t ˆhx t v, h is constant. In other words, v represents sales rate which is independent of the control u. A graphical representation on how the dual controller can be applied to set promotional spending is described in Fig. 2. We start with deriving the optimal promotional spending using the `Regulation Box' for t ˆ. This amount is applied to the system of equation (that is, to the dynamic relationship between the rate of change in sales the promotional e ort its e ect on sales) to nd the current sales measurement. The resulting measurements of sales rate together with the promotional rate just calculated are applied to the `Estimation Box' to re-estimate the state of the system (sales rates the e ect of the promotional rate) in real time. Then set the promotional

8 16 G.E. Fruchter / European Journal of Operational Research 13 (21) 99±11 Fig. 2. The dual controller to set promotional spending. rate in real time by using the `Regulation Box' the cycle is repeated. Next we illustrate the method developed in the previous sections to the above marketing problem. Consider system (2a) (2b) A ˆ ; B ˆ 1 E ˆ h Š: 23 Assume also that Q T in the cost function is given by Q T ˆ qt : 24 Let n ˆ x kš. In terms of x k, Eqs. (14), (15) (18) will become _g 1 ˆ ; g 1 T ˆf ; 25 _g 2 ˆ ug 1 ; g 2 T ˆ; 26 f ˆ q T x T ; u ˆ kg 1 _x ˆ uk; x ˆ^x h p 11 g 1 p 12 g 2 Š; In (23) (24) h q T are constants that can be determined from the history of a similar product situation. For nding the function u in (2), we use Eqs. (14), (15) (18) (or (15) (19)), with the initial values ^n, P. _k ˆ ; k ˆ^k h p 21 g 1 p 22 g 2 Š: Integrating (25) we obtain g 1 t ˆf 3 31

9 G.E. Fruchter / European Journal of Operational Research 13 (21) 99±11 17 k t ˆk : Substituting (31) (32) in (28), we obtain u t ˆ k f : Integrating (26) by considering (33), we obtain g 2 ˆ k f 2 T : Considering (31) (34), the initial conditions in (29) (3) will become x ˆ^x hp 11 f hp 12 k f 2 T k ˆ^k hp 12 f hp 22 k f 2 T : Integrating (29), by considering (32) (33), we obtain x T ˆ k 2 ft x : Substituting (37) in (27), we obtain h f q T k i 2 ft x ˆ : Evaluating (38), considering (35) (36), we obtain that f is the solution of the following 5th order polynomial: Considering (36) (33), we conclude with u ol ˆ u ^n ; P ; T ; t ˆ ^k hp 12 f Šf 1 hp 22 f 2 T ; 4 f is a solution of the 5th order polynomial (39): The closed-loop control strategy will become u ˆ ^k t hp 12 t f Šf 1 hp 22 t f 2 T go : 41 Remark 3. It is interesting to note that a more general case, A ˆ a1 ; 42 a 2 leads to the following result, see Appendix A: u 12 t f ˆ ^b ^k t hp ^ Šf ^ exp a 1 hp 22 t ^b 2 f ^2 2 a 1 tš; 43 ^T go ^b ˆ exp a 2 t ; ^T go ˆ exp 2a 1 t 44 exp 2 a 2 a 1 T Š exp 2 a 2 a 1 tš ; 2 a 2 a 1 45 U f ˆX5 a i f i ; iˆ a ˆ q T x ; 39 ^f ˆ f exp a 1 ^T go ˆ f exp a 1 ^T go T Š: 46 Such a result is useful for a more general model. 7. Conclusions a 1 ˆ 1 q T T ^k 2 hp Š; 11 a 2 ˆ htq T 2p 22 ^x 3p 12 ^k Š; a 3 ˆ 2hT p 22 hq T p 2 12 p 11 Š; a 4 ˆ ht 2 p 22 q T p 22 ^x Š p 12 ^k Š; a 5 ˆ ht 2 p 22 p 22 hq T p 2 12 p 22 Š: We showed that in case of bilinear systems, control estimation could not be designed separately. The estimation (which means learning the uncertainty k) at every instant is a ected by the control; therefore the control performs a dual role of regulation learning. Finding an optimal adaptive control (control which estimates) for bilinear systems is equivalent to solving a dual

10 18 G.E. Fruchter / European Journal of Operational Research 13 (21) 99±11 control problem. We found an analytic closedform solution in a deterministic way. This approach can be viewed as a new way of solving understing adaptive control. The most interesting feature of this work is that the problem is solvable analytically. This is in contrast to stochastic optimal control approaches only approximations are feasible. The analytic solution has a great contribution to the understing of how an optimal dual-nonlinear controller works. From the illustrating example it follows that the cost function may have more than one local minima depending on the number of the solutions of the related ( fth order in this particular case) polynomial equation. The desired solution is associated with the minimum cost with respect to the controller. As the data force the estimates pseudo-error variables, the desired solution can change. Thus, high control, which may be desirable for learning the parameters, can change to low control, that is desirable for keeping the low cost. Multiple minima in cost function were also mentioned in Sternby (1976) for a different approach. To motivate the use of the dual control scheme, we show how it can be applied to set promotion rates for a new product, when little is known on the e ects of promotional spending on sales rates when they change with time. In order to determine promotional expenditures, most of the marketing researches measured the e ectiveness of the promotion (such as advertising) on sales performance by using regression analysis with available data. When regression is used, it is usually applied to promotional expenditure data that were previously determined with the hope of maximizing pro t or sales performance, but not to improve the estimation of the sales±promotion relationship. The proposed adaptive control scheme may o er some more powerful results in such marketing operations. Acknowledgements The author wishes to thank Professor Yoram Halevi for reading this paper for his valuable comments. Appendix A. Derivation of (43)±(46) (Remark 3) Considering (42), Eqs. (14), (15) (18) will become _g 1 ˆ a 1 g 1 ; g 1 T ˆf ; A:1 _g 2 ˆ ug 1 a 2 g 2 ; g 2 T ˆ; A:2 f ˆ q T x T ; u ˆ kg 1 _x ˆ a 1 x u k; x ˆ^x h p 11 g 1 p 12 g 2 Š; _k ˆ a 2 k; k ˆ^k h p 21 g 1 p 22 g 2 Š: Integrating (A.1) we obtain g 1 t ˆf exp a 1 T t Š ˆ f exp a 1 tš; f ˆ f exp a 1 T Š k t ˆk exp a 2 t : A:3 A:4 A:5 A:6 A:7 A:8 A:9 Substituting (A.7) (A.9) in (A.4), we obtain u t ˆ k f exp a 2 a 1 tš: A:1 Integrating (A.2) by considering (A.1), we obtain g 2 ˆ k f 2 T ; A:11 T ˆ exp 2 a 2 a 1 T Š 1 : A:12 2 a 2 a 1 Considering (A.7) (A.11), the initial conditions in (A.5) (A6) will become

11 G.E. Fruchter / European Journal of Operational Research 13 (21) 99±11 19 x ˆ^x hp 11 f hp 12 k f 2 T k ˆ^k hp 12 f hp 22 k f 2 T : A:13 A:14 Integrating (A.5), by considering (A.9) (A.1), we obtain The closed-loop control strategy will become u 12 t f ˆ ^b ^k t hp ^ Šf ^ exp a 1 hp 22 t ^b 2 f ^2 2 a 1 tš; A:2 ^T go ^b ˆ exp a 2 t ; A:21 x T ˆ k 2 f T Š x exp a 1 T Š: Substituting (A.15) in (A.3), we obtain f q T k 2 f T x ˆ ; A:15 A:16 ^T go ˆ exp 2a 1 t exp 2 a 2 a 1 T Š exp 2 a 2 a 1 tš ; 2 a 2 a 1 A:22 q T ˆ q T exp 2a 1 T Š: A:17 Evaluating (A.16), considering (A.13) (A.14), we obtain that f is the solution of the following 5th order polynomial: ^f ˆ f exp a 1 ^T go ˆ f exp a 1 ^T go T Š f is the solution of (A.18). References A:23 U f ˆX5 iˆ a ˆ q T x ; a i f i ; a 1 ˆ 1 q T T ^k 2 hp 11 Š; a 2 ˆ ht q T 2p 22 ^x 3p 12 ^k Š; a 3 ˆ 2hT p 22 hq T p 2 12 p 11 Š; a 4 ˆ ht 2 p 22 q T p 22 ^x Š p 12 ^k Š; A:18 a 5 ˆ ht 2 p 22 p 22 hq T p 2 12 p 22 Š: Considering (A.14) (A1), we conclude with u ol ˆ u ^n ; P ; T ; t ˆ ^k hp 12 f Šf 1 hp 22 b 2 f 2 T exp a 2 a 1 tš; A:19 T is given in (A.12), f is a solution of the 5th order polynomial (A.18). Ascher, U.M., Matteij, R.M.M., Russell, R.D., Numerical Solution of Boundary Value Problems for Ordinary Di erential Equations. Prentice-Hall, Englewood Cli s, NJ. Astrom, K.J., Wittenmark, B., Adaptive Control. Addison-Wesley, Reading, MA. Banks, S., Experimentation in Marketing. McGraw-Hill, New York. Basar, T., Bernhard, P., H 1 ± optimal control related minimax design problems A Dynamic Game Approach. Birkhauser, Boston. Besoussan, A., Van Schuppen, J.H., Optimal control of partially observable stochastic systems with an exponentialof-integral performance index. SIAM Journal of Control Optimization 23 (4), 599±613. Bryson, A.E., Ho, Y.C. Applied Optimal Control. Hemisphere, Washington, DC. Clarke, D.G., Econometric measurement of the duration of the advertising e ect on sales. Journal of Marketing Research 13, 345±357. Deissenberg, Ch., Stoppler, S., Optimal information gathering planning policies of pro t-maximizing rm. Policy Information 7, 49±75. Eastlack, J.O., Rao, A.G., Modeling response to advertising price changes for ÔV-8Õ cocktail vegetable juice. Marketing Science 5 (3), 245±259. Erickson, G.M., Empirical analysis of closed-loop duopoly advertising strategies. Management Science 38, 1732± Feldbaum, A.A., Optimal Control Systems. Academic Press, New York.

12 11 G.E. Fruchter / European Journal of Operational Research 13 (21) 99±11 Fruchter, E.G., Kalish, S., Closed-loop advertising strategies in a duopoly. Management Science 43, 54±63. Lambert, J.D., Computational Methods in Ordinary Di erential Equations. Wiley, New York. Little, J.D.C., Aggregate advertising models: The state of the art. Operations Research 27 (4), 629±667. Mahajan, V., Muller, E., Advertising pulsing policies for generating awareness or new products. Marketing Science 5 (2), 86±16. Neck, R., Stochastic control theory operational research. European Journal of Operational Research 17, 283±31. Pekelman, D., Tse, E., 198. Experimentation budgeting in advertising: An adaptive control approach. Operations Research 28 (2), 321±347. Rao, A.G., 197. Quantitative Theories in Advertising. Wiley, New York. Sternby, J., A simple dual control problem with an analytical solution. IEEE Transactions on Automatic Control, 84±844. Whittle, P., Risk-sensitive linear/quadratic Gaussian control. Advance in Application Probability 13, 764±777. Whittle, P., 199. Risk-Sensitive Optimal Control. Wiley, New York. Whittle, P., A risk-sensitive maximum principle: The case of imperfect state observation. IEEE Transactions on Automatic Control 36 (7). Winer, R.S., Moore, W.L., Evaluating the e ects of marketing-mix variables on br positioning. Journal of Advertising Research 29 (1), 39±45.