Online homotopy algorithm for a generalization of the LASSO

Transcription

1 This article has been acceted for ublication in a future issue of this journal, but has not been fully edited. Content may change rior to final ublication. Online homotoy algorithm for a generalization of the LASSO A. Hofleitner, T. Rabbani, L. El Ghaoui and A. Bayen Abstract The LASSO is a widely used shrinkage method for linear regression. We roose an online homotoy algorithm to solve a generalization of the LASSO in which the l regularization is alied on a linear transformation of the solution, allowing to inut rior information on the structure of the roblem and to imrove interretability of the results. The algorithm takes advantage of the sarsity of the solution for comutational efficiency and is romising for mining large datasets. I. INTRODUCTION AND RELATED WORK Least-squares regression with l -norm regularization is known as the LASSO algorithm []. It has generated significant interest in the statistics [], [], signal rocessing [3], [4], [5] and machine learning [6], [7] communities, in articular for estimation roblems. Adding a l -enalty usually leads to sarse solutions, which is a desirable roerty used to achieve model selection, data comression, or to obtain interretable results. The LASSO can be solved using interior-oint methods [8], iterative thresholding algorithms [9], [0], feature-sign search [], bound otimization methods [], incremental methods [3] or gradient rojection algorithms [4]. Homotoy algorithms comute the regularization ath [5], [6] or erform online udates [7], [8], [9]. They are articularly efficient when the solution is very sarse [0], []. The article extends the results of [8], [9] with the following contributions: (i) the algorithm udates the solution as a new batch of observations is received (revious work only considered udates with one measurement at a time), (ii) the online algorithm solves the LASSO when an affine transformation of the estimate is sarse. Problem statement. At estimation ste n, we are given a set I n of training examles or observations (y i,a i ) R R m, i I n. We wish to fit a linear model to estimate the resonse y i as a function of x R m. A linear function of the solution, K x, with K R k m is exected to be sarse, reresenting inherent structure of the roblem or trend filtering [], [3]. To achieve this roerty, we add an l enalty on K x and solve the following otimization roblem min xr m n i= (a T i x y i ) + µ n K x () The regularization arameter µ n may deend on the number of measurements I n. For examle, we can choose µ n = I n µ 0 as in [8] or µ n = I n µ 0 as in [4]. Both the deendency of Ph.D. candidate, Electrical Engineering and Comuter Science, UC Berkeley, CA. Corresonding author (aude@eecs.berkeley.edu). Ph.D. candidate, Mechanical Engineering, UC Berkeley, CA, trabbani@berkeley.edu Associate rofessor, Electrical Engineering and Comuter Science, Industrial Engineering and Oerations Research, UC Berkeley, CA, elghaoui@berkeley.edu Associate rofessor, Electrical Engineering and Comuter Science, Civil and Environmental Engineering, UC Berkeley, CA, bayen@berkeley.edu µ n on I n and the arameter µ 0 are chosen using a validation metric, comuted on a dataset which is not used to train the model. The validation rocedure is described in more details in the result section (Section IV). It chooses a trade-off between the structure imosed by the regularization, and the fit to the data. After receiving new observations, the algorithm udates the solution of () without having to comletely re-solve the roblem. As done in the Elastic Net [5], we investigate the addition of an l regularization term to () to imrove estimation caabilities by leveraging rior information ˆx on the value of the solution. Organization of the article. Section II reviews an existing homotoy algorithm which solves the LASSO recursively [8], [9]. Section III resents the extension of the algorithm to udate the solution with observations and a l enalization on a linear function of the solution. We aly the algorithm on a traffic estimation roblem from streaming robe data in Section IV and discuss ossible extensions of this work in Section V. II. THE LASSO PROBLEM The LASSO roblem [] is defined as follows: x = arg min Prerint submitted to IEEE Transactions on Automatic Control. Received: Aril 9, 03 0:3:5 PST Coyright (c) 03 IEEE. Personal use is ermitted. For any other uroses, ermission must be obtained from the IEEE by ing ubs-ermissions@ieee.org. xr m n i= (a T i x y i ) + µ n x. () There is a global minimum at x if and only if the subdifferential of the objective function at x contains the 0-vector. The subdifferential of the l -norm at x is the following set 8 ( ) < 9 = kxk = : v v Rm : i = sgn(x i ) if x i > 0 v i [,] if x i = 0 ; where sgn( ) is the sign function. Let A R In m be the matrix whose i th row is equal to a T i, and let y =(y i) T ii n be the vector of resonse variables. The otimality conditions for () are given by A T (Ax y)+µ n v = 0, v kxk. We define the active set a (res. non active set na) as the set of indices reresenting non-zero (res. zero) elements of x. The matrix A a (res. A na ) is a selection of the columns of A in a (res. in na). The non-zero coordinates of x are in x a and x na is the 0-vector. The index a i (res. na i ) references the i th coordinate of the active (res. non active) set. Since v kxk, v ai = sgn(x ai ) and v nai [,]. If the solution is unique, A T a A a is non-singular ; we rewrite the otimality conditions as x a =(A T a A a ) (A T a y µ n v a ) and µ n v na = A T na(a a x a y). If we know the active set and the signs of the coefficients of the solution, thus the vector v a, we can comute the solution x in closed form. When observations come sequentially, a homotoy algorithm [8], [9] solves the LASSO roblem recursively by considering the following roblem: x(t, µ)=arg min xr m A ta T n+! x y ty n+! + µ kxk. Adding (res. removing) a oint is equivalent to comuting the ath from t = 0 to t = (res. from t = to t = 0). Varying The Elastic Net [5] ensures the uniqueness of the solution without requiring A T A to be non-singular.

2 This article has been acceted for ublication in a future issue of this journal, but has not been fully edited. Content may change rior to final ublication. the regularization arameter is equivalent to comuting the ath from µ = µ n to µ = µ n+. III. RECURSIVE LASSO WITH NEW OBSERVATIONS, l AND LINEAR l REGULARIZATIONS We consider a least square estimation roblem, for which we want a linear transform of the solution, K x for K R k m, to be sarse. We udate the estimation as we receive new observations (y new,a new ) R R m. We assume a riori information (e.g. historical estimate) ˆx R m on the solution, which is used as additional regularization when the matrix A is not full column rank or is ill conditioned (see [5] for details). We assume that the matrix K is full row rank, which is the case for numerous alications including total variation regularization. Each row of K corresonds to an information on the sarsity structure of the solution. We define K R m k m such that K =(K T KT )T is non singular. For examle, K is such that the columns of K T form a basis for the null-sace of K. We define a change of variable z = Kx, new data matrices B = AK, B new = A new K and ẑ = K ˆx. We roose an algorithm that udates the solution z of min zr m B tb new! z y ty new! +µ (I k 0 k m k ) z + l z ẑ. (3) as we (i) vary t to add or remove observations and (ii) vary µ to change the weight of the l regularization. The l enalization is on the first k coordinates of z, denoted regularized indices. The last m k indices are in the active set and are referred to as the non-regularized indices. A. Add observations At t = 0, we know the solution z(0, µ n ) and thus the active set and signs of the regularized indices of z. Let v ai be the sign of z ai (0) for the regularized indices and define v ai = 0 for the non-regularized indices. The data matrices with the new observations are indicated with a tilde: B =(B T B newt ) T and ỹ =(y T y newt ) T. The otimality conditions of (3) read B T a ( B a z a (t) ỹ)+(t )B newt a (B new a z a (t) y new ) +µ n v a + l(z a (t) ẑ a )=0 (4) B T na( B a z a (t) ỹ)+(t )B newt na (B new a z a (t) y new ) +µ n w na (t) lẑ na = 0 (5) where w na (t) is a vector with coordinates in [,]. We notice that, at t = 0, z a ( ) and w na ( ) are continuous in t. Let t to be the largest t [0,] such that: (i) for all t [0,t ), for all i in the regularized indices, sgn(z a (t)) = sgn(z a (0)) and (ii) for all t [0,t ), for all i in the non-active set, w nai (t) <. On this interval, v ai is the sign of z ai (t) and Equations (4-5) are valid. We comute Q =( B T a B a + li a ) from its revious value without the new observations using the Woodbury matrix identity ( rank udate). We define z a = Q( B T a ỹ + lẑ a µv a ) and a = t. We consider the singular value decomosition of B new a QB newt a = G T SG and define the rotated data B new = GB new and y new = Gy new,, the rotated error E = B new a z a y new and U = QB newt a. Proosition (Solution ath as we add observations): For t [0,t ), z a ( ) is continuous in t and given by z a (t)= z a (t )U I +(t )S E (6) Let t 0 be the smallest t [0,] such that a coordinate of z a (t) equals zero, t + (res. t ) the smallest t [0,] which sets a coordinate of w na (t) to (res. to -). The transition oint t is defined as t = min(t 0,t +,t ) and can be comuted by solving -degree olynomial equations on a bounded interval. Proof: For t [0,t ), we use (4) and the Woodbury matrix identity to write (Q +ab newt a as) GB new a Q. It follows that: z a (t)= z a au(i + as) B new a z a + a Q z a (t)= z a z a (t)= z a au(i + as) GB new a Q B new a ) as Q au(i + au(i + as) B new a z a + a Uy new au(i + as) Sy new au(i + as) B new a z a + au(i + as) y new Prerint submitted to IEEE Transactions on Automatic Control. Received: Aril 9, 03 0:3:5 PST Coyright (c) 03 IEEE. Personal use is ermitted. For any other uroses, ermission must be obtained from the IEEE by ing ubs-ermissions@ieee.org. B newt a y new which roves (6). The comutation of t 0, t + and t is given by Lemma and. We denote by U i, j the element of U on line i and column j and by U i the i th line of U, s i is the i th singular value in S and E i is the i th coordinate of E. Lemma (Comutation of t 0 ): Let ta 0 i be the smallest value of t [0,] which sets the i th coordinate of qz a (in the regularized indices) to zero. It is given by ta 0 i = aa 0 i + where a 0 a i is the smallest real valued solution in the interval [-,0] of the degree olynomial equation in a, which can be solved numerically: 0 = z ai ( + as l ) a U i, j Ē j ( + as l ) If the olynomial equation does not have real valued solutions in [-,0], we set t 0 a i =. It follows that t 0 is the smallest value of t 0 a i in the interval [0,]. Proof: Setting the i th coordinate of z a to zero in (6), we have 0 = z ai au i (I + as) E 0 = z ai a U i, j Ē j + as j 0 = z ai ( + as l ) a U i, j Ē j ( + as l ) We denote by c i the i th column of B na, d i is the i th row of B new na and d i, j the element of B new na on the i th row and j th If no such t exists, we set t 0 (res. t + and t ) to.

3 This article has been acceted for ublication in a future issue of this journal, but has not been fully edited. Content may change rior to final ublication. column. We also denote by f i the i th element of B T naẽ lẑ na and ẽ = B a z a ỹ. Lemma (Comutation of t + and t ): The smallest value of t that sets the i th coordinate of w na to (res. q to -) is denoted t na + i (res. t nai ). It is given by t na + i = a na + i + q (res. t nai = a nai + ) where a na + i (res. a na + i ) is the smallest real valued solution in the interval [-,0] of the degree olynomial equation in a + (res. in a ): ( µ f i ) (µ f i ) ( + a + s l )=a + E j (d i, j ( + a s l )=a E j (d i, j B a U j )( + a + s l ) B a U j )( + a s l ) If the olynomial equation does not have real valued solutions in [-,0], we set t + na i = (res. t nai = ). It follows that t + (res. t ) is the smallest value of t + na i (res. t nai ) in the interval [0,]. Proof: From (6), we notice that B new a z a (t) y new = B new a z a ag T (I + as) E y new = G T E ag T S(I + as) E = G T (I + as) E We also have B a z a (t) ỹ = ẽ a B a U(I + as) E We rewrite (5) as This roves that z nai takes ositive value and adding the index to the active set rovides a solution, thus the solution (strict concavity). Algorithm udates the solution when t varies from t = 0 to t =. The same algorithm is relevant to remove observations by finding the transition oints as t decreases from to 0. Algorithm Udate of the solution as we add observations Initialize the active set a, non active set na and signs of the regularized indices v a. t = 0 while t < do Comute t 0, t + and t as the smallest value of ta,i 0, t+ na,i and t na,i in (t,] (Lemma -). t = min(t 0,t +,t ) if t > then break; else Udate the active set and sign of the regularized indices according to the transition oint. end if Udate the matrix Q to account for the udated active set (rank udate). end while 0 = B T na(ẽ a B a U(I + as) E)+µw T na lẑ na µw na (t)= B T naẽ lẑ na + ab newt na G T (I + as) E + a(b newt na B T na B a U)(I + as) E We obtain the values of t na + i (res. t nai ) by solving the degree olynomial equation in a + (res. a ) on the interval [-,0]: ( µ f i )( + a + s l )=a + E j (d i, j (µ f i )( + a s l )=a E j (d i, j B a U j )( + a + s l ) B a U j )( + a s l ) B. Udate the regularization arameter The comutation of the regularization ath is detailed in [6] and in [5] for the Elastic Net. To solve the roblem of interest 3, it is necessary to define the non-regularized indices, as done in Section III-A and set v ai = 0 for these indices. With this convention, we refer the reader to [6] and [5] to comute the solution of (3). Remark (Leveraging the sarsity structure): We efficiently udate Q when the active or non active set change or when we add/remove observations with low rank udates. We actually udate the Cholesky factorization of Q which rovides better numerical stability to the algorithm than udating Q directly [6]. Remark (Comlexity): The comlexity of the algorithm deends on the number of transitions and the size of the active set. The theoretical bound on the number of transitions is 3 k, where k is the number of rows of K. In ractice, it is much smaller because successive estimates are exected to have a similar suort. Lemma 3 (Udate of the active set): When we reach a transition oint, the active set and signs of the regularized indices are udated as follows: (i) if t = t 0, we remove the corresonding coordinate from the active set, (ii) if t = t + (res. t = t ), we add the coordinate to the active set and set its sign to ositive (res. to negative). Proof: If t = t 0, let a i be such that z ai (t )=0. The IV. LARGE SCALE TRAFFIC ESTIMATION ON AN ARTERIAL subgradient of (I k 0 k m k )z with resect to the coordinate NETWORK a i is in the interval [-,], hence we remove the coordinate from the active set. A. Exerimental setting If t = t +, let na i be such that w nai (t )=. For t > t, the We aly the algorithm to arterial traffic estimation on a otimality condition for the coordinate na i cannot be satisfied subnetwork of San Francisco, CA totalling more than 800 links with the current active set because w nai is bounded by. If we (.6 kilometers of roadway). Dedicated sensing infrastructure let the coordinate na i of the solution take non-zero values, we is rarely available on arterial networks and we use data can rewrite the otimality condition as f (z a (t))+bz nai (t)=0, collected by the Mobile Millennium system [7] from a fleet where f (z a (t)) < 0 and b is a ositive term which deends of 500 vehicles which reort their location every minute. The on the norm of the column na i of B and B new and on l. duration between two successive location reorts x and x Prerint submitted to IEEE Transactions on Automatic Control. Received: Aril 9, 03 0:3:5 PST Coyright (c) 03 IEEE. Personal use is ermitted. For any other uroses, ermission must be obtained from the IEEE by ing ubs-ermissions@ieee.org.

4 This article has been acceted for ublication in a future issue of this journal, but has not been fully edited. Content may change rior to final ublication. is an observation of the travel time yi on the ath from x to x. We use a conditional random field algorithm [8] to reconstruct the trajectory between x and x. Each trajectory (ath) is converted in a vector ai [0, ]m, where m is the number of links in the network. The jth coordinate of ai, denoted ai, j, is the fraction of the link traveled by the robe vehicle. It is comuted as the distance traveled on the link divided by the length of the link3. In articular, ai, j = 0 if the vehicle did not travel on link j and ai, j = if the vehicle fully traversed link j. The solution xn reresents the average travel time on each link of the network at time t n. We add an l regularization on the satial variations of the travel times for several reasons. First, it imroves estimation caabilities by exloiting a-riori information on the structure of the solution. Traffic signals cause imortant variation on the travel time exerienced on a link and regularization is imortant to revent overfitting. Second, it exhibits the inherent satial structure of traffic by noticing the area where traffic conditions actually change. Finally, the sarse structure of the solution enables an efficient udate of the estimate as new measurements are received. We estimate the average travel times x = K z by solving (3). We use historical mean travel times for the l regularization x. For each new travel time observation, we increase the regularization arameter from In µ0 to In+ µ0 and add the new observation (yn+, an+ ) R Rm. The arameters of the l and l regularizations (resectively µ0 and l ) are chosen via cross-validation as described in the following Section. Observations remain relevant only for a limited eriod of time T 4. When observations become obsolete, the algorithm udates the regularization arameter and removes the old observations (decrease t from to 0). We investigate two otential choices for the full row rank matrix K, which reresents the rior information on the satial structure of the estimate. The first one encourages all the incoming links of an intersection to have the same ace (inverse of velocity), the second one encourages the outgoing links to have the same ace. To each junctions j with n j incoming links (res. outgoing links), corresonds n j rows in K. The kth such row encourages the k and k + incoming (res. outgoing) links of the junction to have the same ace (travel time on the link divided by the length of link). Fig.. Geograhical reresentation of the traffic estimation results. The color of each link varies with the ace (green for small aces, red for large aces). The ins indicate the intersections for which not all outgoing links have the same estimated ace (inverse of the seed). of the regularization matrix K influences the accuracy. The regularization on the outgoing links always rovide better results than the choice of regularization on the incoming links. The algorithm does not rovide an automated way to choose the otimal matrix K even though this is art of current research interest. The results can also be reresented as a traffic ma with colors reresenting the ace of the vehicles: green for smallest ace i.e fastest seed, red for largest ace. We indicate the intersections for which we detect satial variation of the ace by a in. The ins tend to cluster in a few regions of the network, indicating regions with imortant satial variations in the traffic conditions. Imosing and exloiting a sarsity structure on the solution limits the comutational cost of traffic estimation on large networks as the algorithm leverages the sarsity of the solution in the algorithm. The number of transition oints and active indices remain small throughout the algorithm with an average of 0.5 transition oints er estimate udate (addition of new data oints and variation of the regularization arameter) and 0 active regularized indices for a network with 85 links. B. Validation framework At time tn we comute the estimate xn corresonding to the observations in In. We comute the rediction error en = an+ xn yn+ and analyze the effect of the choice of the arameters l and µ0 as well as the choice of matrix K in Figure. The numerical results indicate that both the l and l regularization imrove the results for a wide range of l and µ0. As the error is not very sensitive to the choice of these arameters, they can be calibrated off-line using cross-validation. Figure (right) also indicate that the choice 3 The coefficients a i, j can account for the fact that travel time on a fraction of the link does not vary roortionally with the distance traveled as vehicles are more likely to exerience delays close to signalized intersections [9]. 4 Tyically, T is in the order of five to fifteen minutes V. C ONCLUSION AND DISCUSSION The article resents an online-algorithm to udate the solution of linear regression roblems with a large class of l and l regularizations. The l -norm imroves the accuracy and comutational efficiency of the estimation as well as the interretability of the results by exhibiting and exloiting the underlying sarsity structure of the roblem. The l -norm increases the robustness of the estimator and limits numerical issues. The algorithm rovides the ability to (i) imose sarsity on a linear function of the state, (ii) udate the solution online by comuting a homotoy as new measurements are available (or old ones become obsolete). The otential of the algorithm is demonstrated on real-time traffic estimation Prerint submitted to IEEE Transactions on Automatic Control. Received: Aril 9, 03 0:3:5 PST Coyright (c) 03 IEEE. Personal use is ermitted. For any other uroses, ermission must be obtained from the IEEE by ing ubs-ermissions@ieee.org.

5 This article has been acceted for ublication in a future issue of this journal, but has not been fully edited. Content may change rior to final ublication. Average l error Outgoing link regularization µ 0 = 0. µ 0 = 0.8 µ = Value of λ Average l error Outgoing link regularization λ = 0.0 λ = 0.05 λ = 0.0 Incoming link regularization λ = 0.0 λ = 0.05 λ = Value of µ 0 Fig.. Variation of the l error in function of the arameters l (left) and µ 0 (right). The figures indicate the imortance of the additional l regularization to imrove the accuracy of the estimation. roblem from streaming robe vehicle data in large urban networks. It rovides accurate estimation caabilities and a better understanding of the satial variations of traffic across the network. ACKNOWLEDGEMENTS L. El Ghaoui and T. Rabbani are suorted in art by the National Science Foundation, via Grants SES and CMMI The authors would like to thank Timothy Hunter for roviding the filtered robe trajectories from the raw measurements of the robe vehicles used to roduce the numerical exeriment. REFERENCES [] R. Tibshirani, Regression shrinkage and selection via the Lasso, Journal of the Royal Statistical Society. Series B (Methodological), , 996. [] Y. Dodge, Statistical data analysis based on the l -norm and related methods. Birkhauser, 00. [3] R. Baraniuk, Comressive sensing, IEEE Signal. Proc. Mag., vol. 4, no. 4,. 8, 007. [4] E. Candès, Comressive samling, in Congress of Mathematicians, vol. 3, 006, [5] J. Fuchs, On sarse reresentations in arbitrary redundant bases, IEEE Transactions on Information Theory, vol. 50, no. 6, , 004. [6] I. Guyon and A. Elisseeff, An introduction to variable and feature selection, The Journal of Machine Learning Research, vol. 3,. 57 8, 003. [7] A. Ng, Feature selection, l vs. l regularization, and rotational invariance, in st International Conference on Machine learning. ACM, 004,. 78. [8] S. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, An Interior- Point Method for Large-Scale l -Regularized Least Squares, vol., no. 4, , 008. [9] I. Daubechies, M. Defrise, and C. De Mol, An iterative thresholding algorithm for linear inverse roblems with a sarsity constraint, Communications on Pure and Alied Mathematics, vol. 57, no., , 004. [0] J. Friedman, T. Hastie, H. Höfling, and R. Tibshirani, Pathwise coordinate otimization, The Annals of Alied Statistics, vol., no., , 007. [] H. Lee, A. Battle, R. Raina, and A. Ng, Efficient sarse coding algorithms, Advances in Neural Information Processing Systems, vol. 9,. 80, 007. [] M. Figueiredo and R. Nowak, A bound otimization aroach to wavelet-based image deconvolution, in International Conference on Image Processing, vol.. IEEE, 005. [3] D. Bertsekas, Incremental gradient, subgradient, and roximal methods for convex otimization: A survey, Otimization for Machine Learning,. 85, 0. [4] M. Figueiredo, R. Nowak, and S. Wright, Gradient rojection for sarse reconstruction: Alication to comressed sensing and other inverse roblems, vol., no. 4, , 008. [5] M. Osborne, B. Presnell, and B. Turlach, A new aroach to variable selection in least squares roblems, IMA journal of numerical analysis, vol. 0, no. 3,. 389, 000. [6] B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, Least angle regression, The Annals of Statistics, vol. 3, no., , 004. [7] M. Salman Asif and J. Romberg, Dynamic udating for l regularization, IEEE Journal of Selected Toics in Signal Processing, vol. 4, no., , 00. [8] P. Garrigues and L. El Ghaoui, An homotoy algorithm for the Lasso with online observations, in Neural Information Processing Systems, vol., 008. [9] A. Hofleitner, L. E. Ghaoui, and A. Bayen, Online least-squares estimation of time varying systems with sarse temoral evolution and alication to traffic estimation, in 50th Conference on Decision and Control. IEEE, Dec. 0. [0] I. Drori and D. Donoho, Solution of l Minimization Problems by LARS/Homotoy Methods, in International Conference on Acoustics, Seech and Signal Processing, vol. 3. IEEE, 006. [] I. Loris, On the erformance of algorithms for the minimization of l -enalized functionals, Inverse Problems, vol. 5, no. 3, , 009. [] F. Bach, R. Jenatton, J. Mairal, and G. Obozinski, Convex otimization with sarsity-inducing norms, Otimization for Machine Learning,. 9 53, 0. [3] S. Kim, K. Koh, S. Boyd, and D. Gorinevsky, l trend filtering, SIAM Rev., vol. 5, no., , 009. [4] K. Knight and W. Fu, Asymtotics for lasso-tye estimators, Annals of Statistics, , 000. [5] H. Zou and T. Hastie, Regularization and variable selection via the elastic net, Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 67, no., , 003. [6] G. Golub and C. Van Loan, Matrix comutations. Johns Hokins Univ Press, 996, vol. 3. [7] A. Bayen, J. Butler, A. Patire. et al, Mobile Millennium final reort, University of California, Berkeley, UCB-ITS-CWP-0-6, Tech. Re., 0. [8] T. Hunter, T. Moldovan, M. Zaharia, S. Merzgui, J. Ma, M. J. Franklin, P. Abbeel, and A. M. Bayen, Scaling the Mobile Millennium system in the cloud, in nd ACM Symosium on Cloud Comuting, ser. SOCC, vol. 8. ACM, 0,. 8. [9] A. Hofleitner, R. Herring, P. Abbeel, and A. Bayen, Learning the dynamics of arterial traffic from robe data using a dynamic bayesian network, IEEE Transactions on Intelligent Transortation Systems, 0. Prerint submitted to IEEE Transactions on Automatic Control. Received: Aril 9, 03 0:3:5 PST Coyright (c) 03 IEEE. Personal use is ermitted. For any other uroses, ermission must be obtained from the IEEE by ing ubs-ermissions@ieee.org.