Real-time Pricing Leading to Optimal Operation under Distributed Decision Makings

Transcription

1 Real-time Pricing Leading to Optimal Operation under Distributed Decision Makings Kenji Hirata, João P. Hespanha, and Kenko Uchida Abstract This paper investigates the optimal regulation problem with the steady-state constraints under distributed decision makings, where each agent is allowed to detere its own optimal set-point according to an individual profit. On the other hand, the utility, which corresponds to an individual public commission, tries to realize a socially optimal solution that fulfills steady-state constraints. In order to align the individual decision making of each agent with the socially optimal solution, the utility is allowed to provide additional price, which corresponds to tax or subsidy for the agent. This paper proposes a real-time pricing strategy of the utility. The proposed pricing strategy is heuristic, but we show that the resulting closed-loop system is stable, if the step size parameter in the real-time pricing strategy is sufficiently small, and realizes tracking to the socially optimal solution. I. INTRODUCTION Control design problems usually consider regulation and tracking of system states with respect to known set-points or trajectories. Optimal set-points or trajectories are typically given or detered a priori by solving an appropriate optimization problem. The optimization problem may incorporate economical efficiency, e.g. current fuel prices, security limits of the plant and operating constraints, e.g. power balance constraints of electricity supply/demand network, associated with a steady-state operation of the system. The control system may be required to follow a time-varying demand, and the optimization problem should be solved again to reflect new demand condition and update optimal set-points or trajectories in real-time. If one considers a large scale control system, electrical power supply/demand network for example, an appropriate optimization problem to detere optimal operating conditions becomes large, and solving the problem will be a non-immediate task, especially in real-time. In addition, the components of the entire large scale control system may be distributed, e.g., distributed generators and consumers over the power supply/demand network, and each generator/consumer has own individual economical profit and want to detere its optimal operating condition according to the This work was supported in part by Japan Science and Technology Agency, CREST, and the Institute for Collaborative Biotechnologies through grant W911NF-9-D-1 from the U.S. Army Research Office. K. Hirata is with the Department of Mechanical Engineering, Nagaoka University of Technology, Nagaoka , Japan. J. P. Hespanha is with the Center for Control, Dynamical Systems, and Computation, and the Department of Electrical and Computer Engineering, University of California, Santa Barbara, CA 9316 USA. K. Uchida is with the Department of Electrical Engineering and Bioscience, Waseda University, Tokyo , Japan. K. Hirata and K. Uchida are with Japan Science and Technology Agency, CREST. optimization of individual profit. A centralized optimization to detere the operating conditions is not realistic. This paper investigates the optimal regulation problem with steady-state constraints under distributed decision makings, where each agent, a component of the entire large scale control system, is allowed to detere its own economically efficient optimal set-point according to its individual profit. On the other hand, the utility, which corresponds to an individual public commission, tries to realize a socially optimal solution that fulfills steady-state constraints. The problem considered in this paper is motivated by price based approaches in power balancing [1], [], [3], [4], [5], [6], [7], [8], and in order to align the individual decision making of each agent with the socially optimal solution, the utility is allowed to provide additional price, which corresponds to tax to the agent or subsidy from the community, and each agent will participate to the community through optimization of the individual profit with the additional price. The problem of selecting an economically efficient operating condition has been considered in [9], [1] and references therein, where the problem is investigated in non-distributed fashion in contrast to the approaches in this paper. A dynamic controller based on the Karush-Kuhn-Tucker optimality conditions has been proposed in [9], and a solution that uses penalty and barrier function to deal with constraints has been considered in [1]. The dynamic KKT controller has also been applied to the DC power flow control problem [11] in [5], where it has also been shown that the dynamics KKT controller can be implemented in a distributed fashion to the DC power flow control problem. The problem investigated in this paper, optimal regulation problem with steady-state constraints under distributed decision makings, considers selections of economically efficient operating condition by multiple agents. We propose a realtime pricing strategy of the utility, which tries to align the distributed decision making of each agent with the socially optimal solution. The resulting entire closed-loop system eventually realizes tracking to the socially optimal solution, where no-one needs to solve a large scale optimization problem to detere the operating conditions nor to consider iterative computations to decide the price. The proposed realtime pricing strategy is heuristic. It combines the gradient method for maximization problem and the dynamics of the multiple agents. But we show that the closed-loop system is stable at least locally, if the step size parameter in the proposed real-time pricing strategy is sufficiently small. This structural condition of stability may be preferred, since the large scale control system involves a huge number of agents,

2 and checking a stability condition numerically may become difficult. Due to space limitation, all proofs of the technical results can be found in Appendix and [1]. II. PROBLEM FORMULATION This section describes the optimal regulation problem with steady-state constraints under distributed decision makings. We list some standing assumptions which will be instrumental in the remaining of this paper. A. Dynamics of each Agent We consider a group of n agents, G i, i N = {1,..., n}. Each agent G i has a group of other agents, who are called as the neighbors of G i, and we denote by N i N the set of indices of the neighbors of G i. The agent G i has interactions between its neighbors and is represented by equation of the form ẋ i (t) = f i (x i (t), z i (t), w i (t), r i (t)) z i (t) = g i (x i (t), z i (t), w i (t)) (1a) (1b) where x i X i R ni denotes the state, r i R mi denotes the reference input, z i R mi denotes the output to be tracked to r i, w i W i R mwi denotes an exogenous input and z i R mi represents interactions between the agents, where we set x i X i = j N i X i and consider z i (t) = g i (x i (t)) (1c) We suppose that each function in (1) is differentiable as is required. Assumption 1: Let w i W i be given. For each r i, i N, there exists unique x i, i N that satisfy = f i (x i, g i (x i ), w i, r i ) (a) r i = g i (x i, g i (x i ), w i ) (b) for all i N, and this equilibrium point of (1) is stable at least locally. We suppose that (1) represents a closed-loop system that equipped with a plant to be controlled and local controllers. Assumption 1 requires that this closed-loop system has already been designed as it is locally stable and realizes error free steady state tracking, lim t z i (t) = r i, for each given w i W i. B. Distributed Deterations of Optimal Set-point For each agent G i, selecting an economically efficient setpoint r i is important. We suppose that each agent G i is allowed to select its own set-point r i. We set an optimization problem for each G i, i N and suppose that the economically optimal set-point is given as its optimal solution. r i J i (w i ; r i ) (3a) subject to h ij (w i ; r i ) j T i = {1,,..., t i } (3b) where, for each given w i W i, J i (w i ; ) : R mi R is strictly convex and differentiable, and h ij (w i ; ) : R mi R is convex and differentiable. We denote by r i, i N the optimal solution to (3). An individual public commission, called utility in the remaining of this paper, also tries to incorporate economic efficiency of the agents, but its most important priority is on fulfillment operating constraints at steady-state such as L(w)z(t) + l w (w) =, where z = [ z1 T z T zn T ] T and w = [ w1 T w T wn T ] W = i N W i. Therefore, the utility tries to detere a socially optimal solution as the optimal solution to the following problem. r J i (w i ; r i ) i N (4a) subject to h ij (w i ; r i ) i N j T i (4b) L(w)r + l w (w) = (4c) where r = [ r1 T r T rn T ] T R m, m = i N m i. The matrices L(w) = [ L 1 (w) L (w) L n (w) ] R l m, L i : W R l mi, i N and l w : W R l define equality constraints that the utility want to satisfy. We denote by ri, i N the optimal solution to (4). Assumption : For each w W, the matrix L(w) has full column rank, i.e., rankl(w) = l for each w W. Assumption means that the representation of the equality constraints in (4c) includes no redundant constraint. Assumption 3: For each given w W, constraints in (4b) and (4c) satisfy Slater s constraint qualification condition. Slater s constraint qualification condition implies that strong duality holds for the problem (4) [13]. Assumption 4: For each given w W, there exists an optimal solution to (4). Due to strict convexity of each J i, the optimal solution to (4) is unique for each w W, if it exists. Each agent G i is allowed to detere its own optimal set-point through the optimization of (3). However, it is hard to expect the alignment of r i with r i. In order to align the individual decision making of each agent G i according to (3) with the socially optimal solution to (4), we suppose that the utility is allowed to provide an additional price p i R mi, i N, or, in other word, tax to the agent or subsidy from the community. Therefore, each agent G i participate to the community through the optimization of the following problem. r i J i (w i ; r i ) + p T i r i (5a) subject to h ij (w i ; r i ) j T i (5b) For each given p i, we denote by ri (p i) the optimal solution to (5). Under the above formulations, we consider the feedback interactions between the agents G i, i N and the utility. The utility utilizes the output data z i (t) from each agent G i and tries to detere and provide the price p i (t) in real-time. The agent G i deteres its own input r i (t) as the solution to (5) in real-time according to the provided price p i (t). The optimal regulation problem with steady-state constraints under distributed decision making which will be considered in this paper now can be stated as follows (see also the following Fig. 1): design a real-time pricing strategy of the utility that deteres p i (t) by utilizing output data

3 z i (t) of (1), and eventually realizes ri (p i(t)) ri for all i N. III. PRICING STRATEGY This section investigates steady-state optimality only, and we provide a pricing strategy of the utility that aligns the distributed decision making ri of the agent with the socially optimal solution ri. Real-time pricing strategy incorporating dynamics (1) of the agents will be considered in the next section. We start with the Karush-Kuhn-Tucker conditions of (4). J i (w i ; r i ) + L T i (w)λ + h ij (w i ; r i ) µ ij = (6a) j T i h ij (w i ; r i ) µ ij h ij (w i ; r i ) µ ij = (6b) i N j T i L(w)r + l w (w) = (6c) From Assumption 3, the strong duality holds to (4). Thus, the optimal solution to (4) can also be detered as the variables that satisfy the conditions in (6). The following Lemma 1 states that, at least in steadystate, the utility can choose the optimal price p i that realizes ri (p i ) = r i for all i N. Lemma 1: Let w W be given and denote the optimal solution to (4) or, equivalently, variables that satisfy the KKT conditions in (6) as ri, µ ij, i N, j T i and λ. We have ri (p i ) = r i, if the utility provides the price, p i, according to the pricing strategy p i = L T i (w)λ, i N. Rmi IV. GRADIENT BASED REAL-TIME PRICING STRATEGY In the previous Section III, Lemma 1 shows that the utility can choose the optimal price p i that aligns the distributed decision making ri of each agent with the socially optimal solution ri at least in steady-state. The utility need to solve the optimization problem (4) or the KKT conditions in (6) to detere p i. However, theses problems become very large, since there may exist a huge number of agents. Thus, it may be difficult to solve (4) or (6) and provide its solution p i in real-time. This section proposes a real-time pricing strategy of the utility which utilizes output data z i (t) from the agents. The proposed strategy is heuristic, which combines the gradient method for maximization problem and the dynamics of agents. The stability of the resulting closedloop system is unclear and it will be investigated in the next Section V. A. Real-time Pricing Strategy We start with the dual problem of (4) max λ r J i (w i ; r i ) + λ T (L(w)r + l w (w)) h ij(w i;r i) i N j T i i N If we consider only the optimality in steady-state, as similar to the case of Lemma 1, r i will be detered as ri by each agent G i. By substituting this, we have max J i (w i ; ri) + λ T (L(w)r + l w (w)) λ i N where r = [ (r 1) T (r ) T (r n) T ] T. We apply the gradient method to this maximization problem, then we have dλ dτ = ɛ(l(w)r + l w (w)) ɛ > We replace r by its corresponding output z(t) and have a heuristic real-time pricing strategy of the utility as λ(t) = ɛ(l(w)z(t) + l w (w)) ɛ > (7) Fig. 1 shows a schematic block diagram of the proposed closed-loop system that equipped with a gradient based pricing strategy (7) and distributed optimization (5) of each agent G i, where we note that, in the proposed closed-loop system, no-one need to solve a large scale optimization problem nor consider iterative calculations to detere the price. In addition, to detere the price, the utility need not know any specific characteristics, like functions f i or J i for example, of the agent G i. We investigate stability of the proposed closed-loop system in Fig. 1 in the following Section V, and we will see that it is actually stable. Thus, the pricing strategy (7) can align distributed decision making of each agent with the socially optimal solution at least locally. B. Numerical Example This section considers a numerical example, and we will show how the proposed real-time pricing strategy works. We consider the following coupled dynamics of the two agents 1 ẋ i = ωi1 ζ i1 ω i1 k 1 1 x i ωi ζ i ω i k + k 1 zi + ωi1 r i i = 1, k ω i where we set x T 1 = [ x 11 x 1 x 13 x 14 ] R 4, x T = [ x 1 x x 3 x 4 ] R 4, z T 1 = [ x 11 x 13 ], z T = [ x 1 x 3 ], (z 1 ) T = [ x x 4 ] and (z ) T = [ x 1 x 14 ]. We also set each parameter as ω 11 = 1, ω 1 = 14, ω 1 = 16, ω =, ζ 11 =.9, ζ 1 =.85, ζ 1 =.8, ζ =.7, k 1 = 1 and k =. For each G i, i = 1 and, we define the optimization problem in (3) as where R 1 = r i ri T R i r i + a T i r i subject to (r i c i ) T (r i c i ) rci i = 1, [ ] 1 3 a 1 = [ ] 1 R = [ ] 3 a = c T 1 = [ ] r c1 = 5 c T = [ ] r c = 5 [ ] 3 1 The equality constraint in (4c) is set as [L 1 L ]r+l w w =, where each matrices L 1 = [ 1.5 ], L = [ 1.5 ] and l w = 1 are constants. We set W = [ ], and, in the

4 λ λ = ǫ(l(w)z + l w (w)) w z 1 ẋ 1 = f 1 (x 1, z 1, w 1, r 1 ) z 1 = g 1 (x 1, z 1, w 1 ) r 1 J 1 (w 1 ; r 1 ) + p T 1 r 1 r 1 s.t. h 1j (w 1 ; r 1 ) w 1 p 1 L T 1 (w) w z ẋ = f (x, z, w, r ) z = g (x, z, w ) r J (w ; r ) + p T r r s.t. h j (w ; r ) w p L T (w) w z n ẋ n = f n (x n, z n, w n, r n ) z n = g n (x n, z n, w n ) r n J n (w n ; r n ) + p T r n r n n s.t. h nj (w n ; r n ) w n p n L T n(w) w Fig. 1. A schematic block diagram of the distributed optimization and integrate pricing strategy mechanisms. following simulation example, we use ɛ = 1 and consider an exogenous input w as w(t) = Fig. shows resulting time responses of the closedloop system in Fig. 1. Fig. (a) shows the reference input r i, generated as a solution to (5) by each agent G i, as well as, the corresponding output z i, which should track to r i. Fig. (b) shows the resulting time responses of price p i = L i (w)λ, detered by the gradient based pricing strategy (7). Fig. (c) shows the resulting time response of L(w)z(t) + l w (w), and it can be seen that the equality constraint (4c) is fulfilled in steady-state corresponding to each w W. Figs. (d) and (e) show that the resulting time response of h i (w i ; z i ) (not h i (w i ; r i )) in the optimization problems (5), and h i (w i ; z i ) > indicates a violation of the inequality constraint, but it will be eventually fulfilled in each steady-state. Figs (f) and (g) provide another look of time responses in Fig. (a), and each indicates optimal solution to (4) corresponding to each w W. It can be confirmed that the proposed pricing strategy leads the steady-state response of each agent to the socially optimal operating condition. V. STABILITY ANALYSIS This section investigates stability of the closed-loop system in Fig. 1. The pricing strategy (7) heuristically combines the gradient method for the maximization problem to detere λ and dynamics (1) of each agent G i, and the stability of the resulting closed-loop system in Fig. 1 is not clear. The closed-loop system in Fig. 1 includes the optimization problem (5), and stability analysis does not seem to be straightforward. We first, in order to capture the local behavior around the equilibrium point, replace each optimization problem by its gradient in Section V-A. We need to investigate uniqueness of the equilibrium point of this local dynamics and which is considered in Section V-B. Section V- C considers local stability analysis of the closed-loop system, and especially, we show that the resulting closed-loop system in Fig. 1 is stable at least locally for sufficiently small ɛ >. This structural condition for stability is preferred, since we may have a huge number of agents and a numerical analysis for a large scale system becomes difficult. We start this section with the following preliary result. Definition 1: Let w W be given. A pair (x, λ) and r is said to be an equilibrium point of the closed-loop system in Fig. 1 corresponding to w, if = f i (x i, g i (x i ), w i, r i ) = ɛ(l(w)g(x, w) + l w (w)) r i = g i (x i, g i (x i ), w i ) r i = r i(l T i (w)λ) are held for all i N, where g(x, w) = [ g1 T (x 1, g 1 (x 1 ), w 1 ) gn T (x n, g n (x n ), w n ) ] T. The following two statements look almost immediate consequences from Assumptions 1 and 4, but it is important in stability analysis. Lemma : For each given w W, the closed-loop system in Fig. 1 has an equilibrium point (x, λ) and r. One can confirm that the equilibrium point and the socially optimal solution to (4) is actually identical for each given w W. Lemma 3: Let w W be given and a pair (x, λ) and r be an equilibrium point of the closed-loop system in Fig. 1 corresponding to w. The equilibrium point (x, λ) and r is unique. A. Local Behavior of Distributed Decision Makings The closed-loop system in Fig. 1 includes the optimization problem (5), and the stability analysis does not seem to be straightforward and global stability analysis may not be easy. This subsection investigates the local dynamics of the closed-loop system in Fig. 1 around the equilibrium point corresponding to a given w W. We capture the (5) as a

5 r 11, r 1, r 1, r, z 11, z 1, z 1, z p 11, p 1, p 1, p [L 1 (w) L (w)] [ z T 1 zt ]T l w (w) (a) reference signal, r i, detered by the agent G i and the corresponding output, z i. (b) price, p i, detered by the gradient based pricing strategy. (c) a time history of L(w)z l w(w) that indicates equality constraints. 1 1 h 1 (w; z 1 ) 5 1 h (w; z ) 3 6 r 1, z r, z (d) a time history of h 1 (w 1 ; z 1 ) (not h 1 (w 1 ; r 1 )) that indicates inequality constraints for the agent G 1. 9 (e) a time history of h (w ; z 1 ) (not h (w ; r 1 )) that indicates inequality constraints for the agent G r, z (f) trajectories in r 1 /z 1 plane. Each indicates the optimal solution to (4) for a given w r, z 1 1 (g) trajectories in r /z plane. Each indicates the optimal solution to (4) for a given w. Fig.. A simulation example of the distributed optimization and integrate pricing strategy. mapping from the price, p i, to the reference input, r i, and investigate the gradient of this mapping. Let w W be given and denote by ri, µ ij, i N, j T i and λ the corresponding optimal solution to (4) or, equivalently, the variables which satisfy the KKT conditions in (6). We set p i = L T i (w)λ, i N and consider the optimal solution ri (p i ) to (5). We define I i(p i ) T i for each i N, possibly I i (p i ) =, as I i(p i ) = {j T i h ij (w i ; ri (p i )) = }. The set I i(p i ) contains all index j T i such that the inequality constraint in (5b) becomes active with the optimal solution ri (p ). Therefore, we have the optimality conditions as follows J i (w i ; r i ) + p i + h ij(w i ; r i ) µ ij = (8a) j I i(p i ) h ij (w i ; r i ) = j I i (p i ) (8b) We now consider the following assumption. Assumption 5: There exists δ i >, i N such that I i (p i ) = I i(p i + δp i) holds for all δp i U pi, where U pi = {δp R mi δp i δ i }. Since δ i >, i N are possibly sufficiently small, the assumption considers the case such that the constraint h ij that is active with p i still remains active with p + δp i. Under Assumption 5, we have the conditions in (8), which are identically held for all p i {p i }+U p i, thus we consider the derivative of (8) J i + p i + h ij µ ij = p i j I i(p i ) p i h ij = j I i (p i ) By calculating derivatives of each component in the above equation, we obtain J i p i ri + µ h ij ij r j I i(p i ) i + ( ) T hij = I mi (9a) p i We set j I i(p i ) µ ij p i L i (w i ; r i, µ ij ) = J i (w i ; r i ) + p T i r i + h ij = j I i (p i ) (9b) j I i(p i ) h ij (w i ; r i )µ ij and, by multiplying the matrix ( r/ p) T from the right to (9a), we have that ( ) T ( ) T L i ri ri p i ri + = p i p i From the necessary condition for the optimally, we have L i / ri >, and this concludes that = ( ) T L i ri p i p i ri p i We set E i = ( / p i ) T = ( / p i ) and, for a small deviation δp i from p i, have r i + δr i = r i(p i ) E i δp i + O( δp i ) where δr i denotes the small deviation from r i = r (p i ) due to the small deviation δp i from p i.

6 We now investigate numerical procedures to detere the gradient E i = ( / p i ) T. We set ( L i ) T L i ri = ri = Q i > [ h i1 h i h i Ii(p i ) ] = H ri where I i (p i ) denotes the number of elements in the set I i (p i ). We also set variables to be detered as ( ) T ri E i = = p i p i µ i1 µ i µ K i = [ i Ii(p i ) ] p i p i p i From (9), for each 1 j m i, we have the following (m i + I i (p i ) ) equations [ ] [ ] [ ] Qi H ri ( Ei ) (:,j) (( Imi ) Ii(p i ) ((K i ) (j,:) ) T = (j,:) ) T (1) Ii(p i ) 1 H T ri Thus, the gradient E i = ( / p i ) T = ( / p i ) can be numerically detered for each i N. The gradient E i represents the local behavior around the equilibrium point and, eventually, it depends on a given w i W i. We denote as E i (w i ) in the remaining of the paper. B. Uniqueness of the Equilibrium Point Let w W be given and a pair (x, λ) and r be an equilibrium point of the closed-loop system in Fig. 1 corresponding to w. As we have seen in Lemma 3, the equilibrium point is unique with the closed-loop dynamics in Fig. 1, and it is also an equilibrium point of the closed-loop dynamics with the gradient derived in Section V-A. However its uniqueness is not straightforward. We start with the following definition. Definition : Let w W be give and a pair (x, λ) and r be an equilibrium point of the closed-loop system in Fig. 1 corresponding to w. The pair (x, λ) and r is said to be an isolated equilibrium point in local behavior, if = f i (x i + δx i, g i (x i + δx i ), w i, r i + δr i ) (11a) = ɛ(l(w)g(x + δx, w) + l w (w)) (11b) r i + δr i = g i (x i + δx i, g i (x i + δx i ), w i ) r i + δr i = r i(l T i (w)λ) E i (w)l T i (w)δλ (11c) (11d) are held for all i N only for δλ =. Let the pair (x, λ) and r be an isolated equilibrium point in local behavior for a given w W, δλ = implies that δx = and δr =. Therefore, if one considers local dynamics around (x, λ) and r, (δx, δλ) = (, ) and δr = is an unique equilibrium point of the local dynamics. Lemma 4: Let w W be give and a pair (x, λ) and r be an equilibrium point corresponding to w. The following two statements are equivalent: 1) The pair (x, λ) and r is an isolated equilibrium point in local behavior. ) The matrix L(w)E(w)L T (w) R l l is non-singular, where E(w) = block diag(e 1 (w 1 ),..., E n (w n )). Since each matrix E i (w i ) is only positive semi-definite, E i (w i ), there is a possibility that L(w)E(w)L T (w) becomes singular. C. Local Stability of the Closed-loop System This subsection presents two results for local stability of the closed-loop system in Fig. 1. Both results are stated under the condition in the item ) of Lemma 4. Let w W be given and a pair (x, λ) and r be an equilibrium point of the closed-loop system in Fig. 1 corresponding to w W. We consider the linearization of each agent dynamics (1) as δẋ i = a i (w)δx i + a i (w)δx i + b i (w)δr i (1a) δz i = c i (w)δx i + c i (w)δx i i N (1b) where a i (w) = f i / x i, a i (w) = (f i g i )/ x i, b i (w) = f i /, c i (w) = g i / x i, c i (w) = (g i g i )/ x i, and δx i, δx i, δz i and δr i, i N represent small deviation from the equilibrium point, respectively. By appropriately aligning the matrices in (1), we set the matrices A(w), B(w) and C(w) which represent the entire linearized dynamics in (1) for all i N. Together with δ λ(t) = ɛlδz(t), δp(t) = L T δλ(t) and δr = Eδp, the linearized dynamics of the closed-loop system in Fig. 1 is given by [ ] [ ] [ ] δẋ δ λ A(w) B(w)E(w)L = T (w) δx (13) ɛl(w)c(w) δλ where δx = [δx T 1 δx T δx T n ] T and δλ denotes small deviation from λ. We immediately have the following local stability result. Theorem 1: Let w W be given and a pair (x, λ) and r be an equilibrium point of the closed-loop system in Fig. 1 corresponding to w. Suppose that the matrix L(w)E(w)L T (w) is non-singular. The pair (x, λ) and r is a stable equilibrium point of the closed-loop system in Fig. 1 at least locally, if the linear system in (13) is stable. Theorem 1 states general local stability conditions. However, we may consider a large network of the agents, where there are huge number of agents, and checking the stability of a large size matrix in (13) may become difficult. A structural stability condition such as the closed-loop system is locally stable for small ɛ > may be preferred. The following Theorem states that the closed-loop system is actually stable for small ɛ >. Theorem : Let w W be given and a pair (x, λ) and r be an equilibrium point of the closed-loop system in Fig. 1 corresponding to w. Suppose that the matrix L(w)E(w)L T (w) is non-singular. The pair (x, λ) and r is a stable equilibrium point of the closed-loop system in Fig. 1 at least locally, if ɛ > in (7) is sufficiently small. The structural stability condition in Theorem is preferred, since the network may contains huge number of agents and a numerical approach for stability analysis becomes difficult. In a large network, communication delays could arise another issue, and an effect of time-delays due to communication networks in the proposed real-time

7 pricing strategy is studied in [14]. On the other hand, if one considers a small network, like a network so-called microgrid, a numerical computation for stability analysis may be acceptable. In this case, a stability condition in LMIs can be obtained without introducing the gradient of the optimization problem in Section V-A [15]. VI. CONCLUSIONS Motivated by the control problems of distributed systems, this paper investigates the optimal regulation problem with the steady-state constraints under distributed decision makings, where each agent is allowed to detere its own optimal set-point according to an individual profit. On the other hand, the utility, which corresponds to an individual public commission, tries to realize a socially optimal solution that fulfills operating constraints. In order to align the individual decision making of each agent with the socially optimal solution, the utility provides additional price, which corresponds to tax or subsidy for the agent. This paper proposed the real-time pricing strategy of the utility. The proposed pricing strategy is heuristic, but we showed that the resulting closed-loop system is stable at least locally, if the step size parameter of the gradient method is sufficiently small. ACKNOWLEDGEMENT The authors would like to thank the anonymous reviewers for their helpful comments. REFERENCES [1] A. W. Berger and F. C. Schweppe, Real time pricing to assist in load frequency control, IEEE Transactions on Power Systems, vol. 4, no. 3, p. 9/96, [] C. Silva, B. F. Wollenberg, and C. Z. Zheng, Application of mechanism design to electric power markets (republished), IEEE Transactions on Power Systems, vol. 16, no. 4, p. 86/869, 1. [3] F. L. Alvarado et al., Stability analysis of interconnected power systems coupled with market dynamics, IEEE Transactions on Power Systems, vol. 16, no. 4, p. 695/71, 1. [4] R. Wilson, Architecture of power markets, Econometrica, vol. 7, no. 4, p. 199/134,. [5] A. Jokić, M. Lazar, and P. van den Bosch, Real-time control of power systems using nodal prices, International Journal of Electrical Power and Energy Systems, vol. 31, no. 9, p. 5/53, 9. [6] N. Li, L. Chen, and S. H. Low, Optimal demand response based on utility maximization in power networks, in Proceedings of the 11 IEEE Power and Energy Society General Meeting, 11, pp [7] Y. Okajima, T. Murao, K. Hirata, and K. Uchida, Real time pricing and pivot mechanism for LQG power networks, in Proceedings of the 13 IEEE Multi-conference on Systems and Control, Hyderabad, India, 13, pp [8], A dynamic mechanism for LQG power networks with random type parameters and pricing delay, in Proceedings of the 5nd IEEE Conference on Decision and Control, Florence, Italy, 13, pp [9] A. Jokić, M. Lazar, and P. P. J. van den Bosch, On constrained steadystate regulation: Dynamic KKT controllers, IEEE Transactions on Automatic Control, vol. 54, no. 9, p. 5/54, sept. 9. [1] D. DeHaan and M. Guay, Extremum-seeking control of stateconstrained nonlinear systems, Automatica, vol. 41, no. 9, pp , 5. [11] R. D. Christie, B. F. Wollenberg, and I. Wangensteen, Transmission management in the deregulated environment, Proceedings of the IEEE, vol. 88, no., pp ,. [1] K. Hirata, J. P. Hespanha, and K. Uchida, Real-time pricing leading to optimal operation under distributed decision makings, Nagaoka University of Technology, Tech. Rep., 13. [Online]. Available: hirata/guest/acc14/ [13] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge University Press, 4. [14] G. Baba, K. Hirata, T. Murao, and K. Uchida, Optimal operation under distributed decision makings with pricing delay, in Proceedings of the 1st Multi-symposium on Control Systems, 14, pp. 6A3 4, in Japanese. [15] Y. Okada, K. Hirata, and K. Uchida, Real-time pricing leading to optimal operation and its stability analysis, in Proceedings of the 1st Multi-symposium on Control Systems, 14, pp. 6A3 1, in Japanese. [16] J. M. Maciejowski, Multivariable feedback design. Addison-Wesley, Proof: [Theorem ] To simplify the notations, we omit the dependency to w of each matrix. We set the loop transfer function as A BEL T ɛlc I l = ɛ 1 s LM(s)ELT where M(s) = C(sI nx A) 1 B, n x = i N n i, and the matrix A is Hurwitz due to Assumption 1 in which we suppose that (1) represents a locally stable closed-loop system instead a plant to be controlled itself. The set of poles of the loop transfer function ɛ(1/s)lm(s)el T is a subset of the n x -stable poles of the matrix A and l- poles at the origin due to integrators. Let us denote λ A i, i = 1,,... n x the eigenvalues of the matrix A and set ω 1 = i=1,,...nx λ A i. We will pick < ω l < ω 1 and consider the ω l -modified D-contour that has a semicircular indentation into the left half plane with radius ω l at the origin. Since ω l < ω 1, the loop transfer function ɛ(1/s)lm(s)el T has no pole on boundary and inside of the ω l -modified D-contour except l-poles at the origin. We apply the Nyquist stability criteria with the ω l -modified D- contour and will see that, for a sufficiently small ɛ >, the image of det[i l + ɛ(1/s)lm(s)el T ] or, equivalently, the characteristic loci [16] (the graphs of the eigenvalues of I l + ɛ(1/s)lm(s)el T ) encircle the origin l-times counterclock-wise as s goes once around the ω l -modified D-contour, and this will conclude that the closed-loop system with the loop transfer function ɛ(1/s)lm(s)el T and negative feedback has no poles on boundary and inside of the ω l - modified D-contour as well as the closed right half plane. From Assumption 1, the transfer function matrix M(s) satisfies lim ω M(jω) = m lim M(s) = I m (14) s Let us denote λ i (s), i = 1,,..., l the eigenvalues of LM(s)EL T. We also denote λ ni, i = 1,,..., l the eigenvalues of LEL T, where λ ni > since LEL T is positive definite. Because of (14), each λ i (s), i = 1,,..., l satisfies lim λ i(jω) = ω lim λ i(s) = λ ni (15) s The eigenvalues of I l + ɛ(1/s)lm(s)el T are given by 1 + ɛ(1/s)λ i (s), i = 1,,..., l, and we have det[i l + ɛ(1/s)lm(s)el T ] = i=1,,...,l (1 + ɛ(1/s)λ i(s)). In the

8 remaining of this proof, we will see that each graph of 1 + ɛ(1/s)λ i (s), as s goes once around the ω l -modified D- contour, encircles the origin, and this will conclude that the characteristic loci of I l +ɛ(1/s)lm(s)el T, taken together, encircle the origin l-times. 1) Low frequency portion: Let ɛ >. For each i = 1,,..., l, we set h i (s) = 1 + ɛ 1 s λ i(s) and investigate h i (jω) when ω becomes small. We note that, since w >, h i (jω) = (ωh i (jω)), and we have ωh i (jω) = ω + ɛim[λ i (jω)] + j( ɛre[λ i (jω)]) From (15), we further have This concludes that lim (ω + ɛim[λ i(jω)]) = ω lim ( ɛre[λ i(jω)]) = ɛλ ni < ω lim h i(jω) = π ω (16) We next investigate h i (jω) when ω becomes small. Let us consider ω h i (jω) = (ω + ɛim[λ i (jω)]) + (ɛre[λ i (jω)]) and, from (15), we have lim ω ω h i(jω) = ɛλ ni This concludes that h i (jω) will diverge when ω becomes small, thus we have lim h i(jω) = (17) ω From (16) and (17), for any possibly small ɛ a >, and any possibly large R > 1, we can find ω > ω 1 such that h i (jω) π < ɛ a and h i (jω) > R for all ω (, ω ), and all i = 1,,..., l. ) Medium to High frequency portion: We will see that we can pick ɛ >, possibly small, such that Re[h i (jω)] > holds for all ω [ ω, ) and all i = 1,,..., l. We set g m = i=1,,...,l inf Imλ i(jω) ω (, ) where g m, because of (15). We note that, since ω >, Re[h i (jω)] > is equivalent to Re[ωh i (jω)] >. For each ω [ω, ), from the definition of g m, we have Re[ωh i (jω)] = ω + ɛim[λ i (jω)] ω + ɛg m ω + ɛg m for all i = 1,,..., l. In case of g m <, we define ɛ 1 = ( ω )/g m >, and, by setting < ɛ < ɛ 1, we have Re[ωh i (jω)] ω + ɛg m > ω + ɛ 1 g m = (18) for all ω [ ω, ), and all i = 1,,..., l. If we have g m =, we obtain Re[ωh i (jω)] ω + ɛg m = ω > (19) for all ω [ ω, ), all ɛ >, and all i = 1,,..., l. Thus, we set ɛ = 1. From (18) and (19), we set < ɛ < {ɛ 1, ɛ } and conclude that Re[h i (jω)] > for all ω [ ω, ), and all i = 1,,..., l. 3) Semi circular indentation with radius ω l : We will see that, when s moves around the origin, the number of encirclement around the origin made by h i (s) = 1 + ɛ(1/s)λ i (s) is same to the number made by 1 + ɛ(1/s)λ ni. 3a): Let us set λ = i=1,,...,l λ ni Because of (15) and λ i (s) is continuous in s, for any < δ < λ, we can pick < ω 3 < ω such that λ i (s) λ ni < δ () for all s ω 3, and all i = 1,,..., l. We now pick a sufficiently small ω l >. It is possible to find ω 4 > that satisfies ω 3 ω 4 > ω l and λ i (s) λ ni < δ ω l for all s ω 4 (1) ɛ 3b): From (1), for each s ω l and i = 1,,..., l, we have λ i (s) = λ ni + (δ ω l ɛ )u i(s) for some u i (s) 1. We set s = ω l e jθ, θ [ ( 3π)/, ( π)/ ] and we further have h i (ω l e jθ 1 ( ) = 1 + ɛ ω l e jθ λ ni + (δ ω ) l ɛ )u i(θ) = 1 + ɛλ ni ω l e jθ + ( ɛδ ω l 1)e jθ u i (θ) () for some u i (θ) 1, where we denote u i (θ) = u i (s) for s = ω l e jθ. When s = ω l e jθ makes clock-wise movement as θ = ( π)/ π ( 3π)/ the first and second term in () makes counter-clock-wise semicircular movement around 1+ j with the radius (ɛλ ni )/ω l. While, for each θ, the third term belongs to the ball with the radius (ɛδ)/ω l 1. We have ɛλ ni ω l 1 > ɛδ ω l 1 since δ < λ. This concludes that the number of encirclement around the origin made by h i (s) = 1 + ɛ(1/s)λ i (s) is same to the number made by 1 + ɛ(1/s)λ ni. This shows that the characteristic loci of I l + ɛ(1/s)lm(s)el T, taken together, encircle the origin l- times counter-clock-wise and concludes that the closed-loop system with the loop transfer function ɛ(1/s)lm(s)el T and negative feedback has no poles on boundary and inside of the ω l -modified D-contour as well as the closed right half plane.