Robust Estimation of Origin-Destination Matrices

Transcription

1 Uniersity of Kansas From the SelectedWorks of Byron J Gajewski June, 00 Robust Estimation of Origin-Destination Matrices Byron J Gajewski, Uniersity of Kansas Medical Center Laurence R Rilett Michael Dixon, Uniersity of Idaho Clifford H Spiegelman, exas A&M Uniersity Aailable at:

2 Robust Estimation of Origin-Destination Matrices BYRON J GAJEWSKI he Uniersity of Kansas LAURENCE R RILE exas A&M Uniersity and exas ransportation Institute MICHAEL DIXON Uniersity of Idaho CLIFFORD H SIEGELMAN exas A&M Uniersity and exas ransportation Institute ABSRAC he demand for trael on a network is usually represented by an origin-destination (OD trip table or matrix OD trip tables are typically estimated with synthetic techniques that use obsered data from the traffic system, such as link olume counts from intelligent transportation systems (IS, as input A potential problem with current estimation techniques is that many IS olume counters hae a relatiely high error rate he focus of this paper is on the deelopment of estimators explicitly designed to be robust to outliers typically encountered in IS Equally important, standard errors are deeloped so that the parameter reliability can be quantified his paper first presents a constrained robust method for estimating OD split proportions, which are used to identify the trip table, for a network he proposed approach is based on a recently deeloped statistical procedure known in the literature as the L error (L E Subsequently, a closedform solution for calculating the asymptotic ariance associated with the multiariate estimator is deried Because the solution is closed form, the computation time is significantly reduced as compared with computer-intensie standard error calculation methods (eg, bootstrap methods, and Byron Gajewski, Schools of Allied Health and Nursing, 304 SON, Uniersity of Kansas Medical Center, 390 Rainbow Bld, Kansas City, KS Bgajewski@kumcedu 37

3 therefore confidence interals for the estimators in real time can be calculated As a further extension, the OD estimation model incorporates confirmatory factor analysis for imputing origin olume data when these data are systematically missing for particular ramps he approach is demonstrated on a corridor in Houston, exas, that has been instrumented with IS automatic ehicle identification readers INRODUCION he demand for trael on a network is usually represented by an origin-destination (OD trip table or matrix A trip is a moement from one point to another, and each cell, OD kj, in the table represents the number of trips starting at origin k and ending at destination j An OD trip table is a required input to most traffic operational models of transportation systems In addition, demand estimates are useful for real-time operation and management of the system While it would be possible to directly measure the trael olume between two points, it is a ery expensie and time-consuming process to identify the trip matrix for an entire network Consequently, OD trip olume tables are typically estimated with synthetic techniques that use obsered data from the traffic system, such as link olume counts from inductance loops, as input Oer the past 0 years, numerous methodologies hae been proposed for estimating OD moements within an urban enironment (Bell 99; McNeil and Henderickson 985; Cascetta 984 A common approach for estimating OD olumes is based on least squares (LS regression where the unknown parameters are estimated based on the minimization of the Euclidean squared distance between the obsered link olumes and the estimated link olumes Deriing OD trip tables from intelligent transportation systems (IS olume data is the focus of this paper Nihan and Hamed (99 demonstrated that outlier data can hae a significant impact on OD estimate accuracy, but most OD estimation techniques assume that the input data are reliable Furthermore, it has been shown that unless inductance loops are maintained and calibrated at regular interals, the data receied from them are prone to error rates of up to 4% (urner et al 999 Faulty olume counts can occur because of failures in traffic monitoring equipment, communication failure between the field and traffic management, and failure in the traffic management archiing system he first two failure types may be difficult to detect because they occur in isolated detectors Other detector problems include stuck sensors, chattering, pulse breakup, hanging, and intermittent malfunctioning Consequently, there is a need to account for the input error in the estimation process Historically, detector malfunctions hae been addressed by cleaning the datasets prior to the estimation step his is problematic for IS applications, because the data manipulation takes time and the use of these techniques is limited in a real-time enironment Een for off-line applications, cleaning the data is problematic, because it is not always clear which data need to be cleaned and how this can best be accomplished his paper examines a robust approach whereby the data error associated with the faulty input data is accounted for explicitly within the estimation of the OD trip table he deelopment of OD estimators that are robust to detector malfunction has been largely ignored in OD estimation research, with the exception of an approach proposed using a least absolute norm (LAN estimator (Sherali et al 997 LAN is intuitiely more robust to outliers than LS, because the errors are not squared as they are in LS (Cascetta 984; Maher 983; Robillard 975, nor are the data treated as constraints as they are in many maximum entropy and information minimization approaches (Cascetta and Nguyen 988; Van Zuylen 980 his paper follows on their work by using estimators explicitly designed to be robust to outliers typically encountered in an IS enironment Equally important, standard errors will be deeloped so that parameter reliability can be quantified his paper first presents a constrained robust method for estimating OD split proportions, which are used to identify the trip table, for a network he traffic olumes are assumed to come from an Adanced raffic Management System (AMS, and the traditional assumption that the data are reliable is relaxed to allow the possibility of missing or faulty detector data he proposed 38 JOURNAL OF RANSORAION AND SAISICS V5, N/3 00

4 approach is based on a recently deeloped parametric statistical procedure known in the literature as the L error (L E Subsequently, a closed-form solution for calculating the asymptotic ariance associated with the multiariate estimator is deried Because the solution has a closed form, the computation time is significantly reduced as compared with the computer-intensie calculation of the standard errors (eg, bootstrap methods and off-line methods based on the LS (Ashok 996, and therefore confidence interals for the estimators in real time can be calculated As a further extension, the OD estimation model incorporates confirmatory factor analysis (ark et al 00 for imputing olume data when this information is systematically missing for particular ramps While the techniques demonstrated here are motiated using inductance loop data, they are easily generalized to other detectors, models, and data he approach is demonstrated on a corridor in Houston, exas, that has been instrumented with automatic ehicle identification (AVI readers he AVI olumes are used to estimate an AVI OD matrix he benefit to this test bed is that the actual AVI OD table can be identified and, therefore, proides a unique opportunity for directly measuring the accuracy of the different techniques he OD matrix can be obtained using a linear combination of the split proportion matrix and the origin (entrance olume ector It should be noted that because the input olumes are dynamic, the estimated OD matrix is also dynamic Howeer, because the split proportion is assumed constant, the OD matrices by time slice are linear functions of each other Also note that while the method is acceptable for freeway networks with relatiely short trips between interchanges, such as the example in this paper, an application to a larger network would raise the question as to when the trips began or ended Caution should be exercised when applying this method to a large proportion of trips that begin and end during different time periods Basu et al (998 and Scott (999 contributed the idea to the parametric literature L refers to the squared distance between two points he L E was originally used in kernel density estimation (see, eg, errell 990 HEORY he objectie of this paper is to estimate the OD split proportion matrix and its associated distribution properties he split proportion kj is defined as the proportion of ehicles that exit the system at destination ramp j gien that they enter at origin ramp k While the split proportions are estimated using only the olumes obsered at the origins and destinations, the approach can easily be extended to the case where general lane olumes are obsered as well Once the OD split proportion matrix is identified, the OD table can be deried See the appendix for a notation list and all proofs of the theorems Methodology A desirable feature of an OD estimator is the ability to calculate a closed-form limiting distribution his is particularly important for IS applications, so that confidence interals about the estimate can be calculated in real time he estimator will be defined and distributional properties of the estimator will be deried following the steps below Define the multiariate constrained regression model and objectie functions Obtain distributional properties a Stack the ariables to obtain a uniariate constrained regression model b Use the equality constraints to obtain a uniariate unconstrained regression model c Apply asymptotic theory to derie the distributions of the estimated split proportions for the LS and the L E estimation techniques d ransform the model back to its original form Step : Define the Model and Objectie Functions A traffic network can be represented by a directed graph consisting of arcs (or links representing roadways and ertices (or nodes representing intersections here are q origin (entrance ramps and p destination (exit ramps, and the analysis period is broken down into time periods of equal length t Figure shows an example of a traffic system in Houston, exas, along the inbound (east- GAJEWSKI, RILE, DIXON & SIEGELMAN 39

5 FIGURE Eastbound Interstate 0 in Houston, exas N US-90 est bed I-45 US 59 FM 960 I-0 I-0 US 59 SH 88 I-45 En Ex 6 Ex En Ex En 3 Ex 3 En 4 Ex 4 En 5 Ex 5 En miles 365 miles 5 miles 405 miles 45 miles bound Interstate 0 (Katy Freeway corridor and a schematic diagram of the corridor he underlying assumption of the OD model deeloped here is that mass conseration holds In essence, it is assumed that the traffic entering the system also exits the system within a specified time period his assumption is appropriate as long as the time interals are long and/or the traffic is in a steady state For time period t, t,,,, D tj denotes the destination olume at destination ramp j, j,,3,,p and O tk denotes the origin olume at origin ramp k, k,,3,,q he first assumption is that the split proportion kj is constant oer the entire analysis period he second assumption is that the number of ehicles entering the system equals the number of ehicles exiting the system so that the split proportions at each origin during the period sum to kj ( he expectation of the destination olumes during j each time period t is a linear combination of the split proportions weighted by the origin olumes D O + ε ( he term ε is the error Intuitiely, some of the split proportions are not feasible ( kj 0 because of the structure of the traffic network, and this needs to be incorporated in the estimation process Note that in some cases the nonfeasible parameters may simply represent unlikely cases he zeros are exact in the applications presented later It is assumed in this paper that the errors in each column are independently and identically distributed with mean zero and constant ariance his means that the error associated with the olume measurement at destination j is uncorrelated with the olume measurement at destination j Note that all assumptions will be checked in the applications section of the paper 40 JOURNAL OF RANSORAION AND SAISICS V5, N/3 00

6 Because the OD problem structure is implicit rather than explicit, and because it is also oerspecified, the synthetic approaches attempt to estimate the split proportion matrix based on the minimization of an objectie function (Dixon 000 he objectie function in equation is based on an LS approach, which is a common way to estimate the split proportions (Robillard 975 Note that if ε t is distributed normally then the LS is equialent to the maximum likelihood estimator he central hypothesis of this paper is that the LS method is inappropriate for IS applications because of the high error rate of the measured olumes In contrast, the L E objectie function is defined as the integrated squared difference between the true probability density function and the estimated density function herefore, it is theoretically more robust, relatie to the LS, when the data hae outliers such as those caused by detector malfunctions A more specific discussion of this follows As an example, consider a sample of size If the goal is to estimate the location parameter, or the central tendency, O t 0 (suppose 0 is the true unknown split proportion, estimated with, then the minimization of the integrated squared error is shown in equation 3 and deried in the appendix, min F F L E min he normal distribution, D ~ N O, σ, replaces p ( 3 [ f ( D O ] dd tj t j tj f ( D O p tj t j ( tj Ot0 j f D ( D O min min ( F LS tj t j t j function displayed as equation 4, p t j ( 0 tj t j resulting in the objectie he L E minimizes the sum of probability density functions (pdfs while the MLE (or LS mini- ( O σ p min FL E min N Dtj πσ p t j t j, (4 mizes the negatie product of pdfs In the presence of an outlier, the MLE objectie function will multiply a zero and the L E objectie function would add a zero he relatie effect of the outlier on the objectie function is much more seere in the case of the MLE his is because of the multiplicatie effect of the zeros on the MLE objectie function Because IS data often contain outliers, it is hypothesized that the L E objectie function will proide better estimates For a more detailed discussion of the L E properties, see Basu et al (998 or Gajewski (000 When calculating the L E, an estimate of the ariance is required In this paper, the estimate of the ariance is identified using a two-step process First is calculated using the median of the squared residuals so that an initial estimate of the standard deiation, σ, may be obtained by calculating the median absolute deiation (MAD from the set of estimated residuals, where the t th estimated residual is εt D t Ot he median of the squared residuals is used because it has been shown to hae a high breakdown point (Hampel et al 986 In general, the breakdown point is the percentage of outliers in the data at which the estimator is no longer robust (Huber 98 For example, if the breakdown point is 50% then the estimator is robust to datasets that contain less than 50% outliers Steps a b: Stack and Define the Unconstrained Uniariate Regression Model hrough reparameterization, the split proportion model shown in equation can be translated into an unconstrained uniariate regression model he first step is to stack the split proportion matrix, where ec( For example, GAJEWSKI, RILE, DIXON & SIEGELMAN 4

7 he equality constraints are subsequently defined relatie to using G g, where the matrices G p I q M and q g 0 he matrix M (r by qp maps the nonfeasible alues of to zero he alue r is the number of split proportions equal to zero As an example, define where 0 and 0 herefore, the product of the M and matrix maps the nonfeasible elements to zero M where G [ G G ] he first q rows of each G and g are used to normalize the split proportions as shown in equation 5 Gien the constraints defined aboe and the stacked model, the reduced model may be obtained by soling equation 5 Because G is triially full rank (q + r: ( 6 G g G Equation is placed in uniariate form by defining Y to be the stacked columns of D and at the same * time letting X Ip O When equation 6 is substituted into the stacked model the unconstrained model is obtained he substitution is done with the portion of the stacked model that corresponds to the * * * X X X regression parameters If [ ] * * then Y X + X + ε r ( 5 ( Y W +ε ( 7 where * * * Y Y X G g Step c d: Variance Estimates for the Reparameterized Model and Distributions hese next steps produce distributions for the estimators under the reparameterized model, in particular that both L E and LS produce estimators that consistently estimate the true (or p, and that is asymptotically normally distributed, under certain assumptions he asymptotic normality directly produces ariance estimates of he asymptotic properties follow directly from the reparameterized model discussed in Steps a b hen the asymptotic theorems from Huber (98 from the objectie functions (or deriaties of them called M-estimators are applied, written in an unconstrained form he only additional assumptions for these asymptotic results are that the errors in equation hae a mean of zero and a ariance of σ I p and are iid his assumption encompasses many distributions that include heaier tails then the normal distribution he specific calculations for the LS asymptotic ariances are Var( ( U U and, Var where ψ and U are defined in the appendix One result of this is when the errors are normally distributed the ratio of the ariances of the L E estimator and the LS estimator of any element of L E ( is ( W X X G G E , [ ψ ] ( [ ] ( UU E ψ, and the new regression model, which describes in uniariate form the multiariate regression problem, is defined in equation 7 4 JOURNAL OF RANSORAION AND SAISICS V5, N/3 00

8 which says that the ariance under the L E estimator is 54 times as high as the LS estimator herefore, the L E estimator is actually less efficient than its LS estimator counterpart Howeer, by applying a heay-tailed distribution, the result reerses and the L E estimator is better than the LS estimator In the subsequent sections, a simulation demonstrates this result Conclusion of heory he deriation of the asymptotic distributions has shown that explicit closed-form ariance estimates exist and that associated standard errors of the split proportions can be calculated in real time Notice that these theorems can be extended to other types of M-estimators, in addition to the L E, as long as the assumptions are met Note that if the assumptions to derie the closedform standard errors are not appropriate, then bootstrap methods will be required he bootstrap is a computer-intensie procedure that uses resamples of the original data to obtain properties such as standard errors One can use arious algorithms to do this, such as case resampling or model-based resampling (Daison and Hinkley 997 ALICAION est Bed he test bed is an eastbound section of Interstate 0 (I-0 located in Houston, exas, as shown in figure his section of I-0 is monitored as a part of the Houston ranstar ransportation Management Center (MC (operated by the exas Department of ransportation, the Metropolitan ransit Authority of Harris County (MERO, the City of Houston, and Harris County here are six automatic ehicle identification readers located in the test bed As instrumented ehicles pass under an AVI reader, a unique ehicle identification number is recorded and sent to a central computer oer phone lines From this information, the aerage link trael time is calculated and presented to driers through arious traeler information systems It may be seen in figure that the I-0 test corridor is made up of fie AVI links consisting of six origins and six destinations Note that each destination may consist of seeral destination ramps and each origin may consist of seeral origin ramps, because the AVI links are defined by the location of the AVI readers and not the physical geometry of the corridor he OD methodology deeloped in this paper was motiated by the assumption that the MC has access to olume information obtained from point detectors (ie, inductance loops Howeer, the models are tested using data obtained from an AVI system because both AVI olumes and split proportions can be identified his proides a unique opportunity for comparing the accuracy of the OD estimates that are based on olumes obtained at point sources with obsered OD split proportions In this situation, the split proportions are estimated using only the OD olumes from the AVI data Based on a preliminary analysis, 8 days in 996 were used for the analysis (October,, 4, 5, 5 9, 6, 30, and Noember, 6, and 8 he AVI ehicles detected at the origin and destination ramps during the AM peak period (7 am to 9 am were aggregated into 4 olume counts of 30- minute duration herefore, the number of time periods ( is equal to 7, the number of destinations (p is 6, and the number of origins (q is 6 he AVI olumes were used as input to the LS and the L E estimators and the resulting estimates are presented in table he assumption that the split proportion matrix was constant, which was used in the deriation of the LS and the L E estimators, was erified by examining the obsered AVI split proportion differences across all days and all time periods of the study Because these obsered split proportion differences were within 0 for 95% of the estimates, it was judged that the assumption was reasonable for this test case he obsered mean AVI split proportions for the AM peak are shown in table he correlation coefficients between the obsered mean AVI split proportions and the estimated split proportions by the LS and the L E esti- See Dixon (000 for specific data extraction techniques GAJEWSKI, RILE, DIXON & SIEGELMAN 43

9 ABLE OD kj Estimated Split roportions by OD air for the eak eriod (7 9 am kj Key: OD kj Vehicles depart from origin node k and arrie at destination node j kj LS kj L E kj LS kj L E kj Obsered split proportion matrix between origin node k and destination node j Estimated split proportion between origin node k and destination node j using least squares estimator Estimated split proportion between origin node k and destination node j using the L E estimator mators are 096 and 0960, respectiely hese results indicate that both estimators are successful at estimating the OD split proportions Howeer, the mean absolute percent error (MAE between the mean obsered split proportion and the estimated split proportions using the LS and the L E estimators were 445% and 53%, respectiely hese results indicate each method has an approximate error of 50% with respect to indiidual OD split proportion estimates A further analysis of the absolute percent error (AE at the indiidual OD leel showed that the AE decreases as the size of the OD split proportion increases For example, the MAE for OD pairs that hae obsered split proportions in the range of 0 00, , and greater than 048 are 78%, 3% and 9%, respectiely, for the LS estimator he L E analysis had similar results in that the percentage error for cells that hae split proportions in the range of 0 00, , and greater than 048 were 95%, 388%, and 63%, respectiely As would be expected from their objectie functions, the estimators tend to hae more accurate results for the more important OD pairs as measured by split proportion or relatie olume In general, these results are encouraging because the estimates for the OD pairs that hae higher split proportions, which will in all likelihood be the more important ones for IS applications, will tend to be more accurate he asymptotic ariance calculated using the closed-form solutions deeloped in this paper and ariance estimators calculated using a bootstrap technique are shown in table he bootstrap estimate of the standard error was calculated using 999 estimates of the split proportion hese estimates were calculated from realizations produced from resampling the rows of D and O (Daison and Hinkley 997 he AE between the asymptotic standard deiation and the bootstrap standard deiation ranged quite a bit Howeer, the AE tended to be lower for the OD pairs with higher obsered OD split proportions, which was similar to that of the preious analysis For the L E analysis, the AE between the asymptotic standard deiation and the bootstrap standard deiation also ranged quite a bit Similar to the LS analysis, the standard deiation of the AE tended to decrease as the size of the obsered split proportion increased In general the L E asymptotic standard deiation gae better results in 3 of the parameters, as compared with the bootstrap standard deiation the LS produced he asymptotic standard deiation for both the LS and the L E estimators were, on aerage, higher than the bootstrap method and can therefore be used as a conseratie estimate of the standard deiation in real-time applications Based on the estimated split proportions and the ariance shown in table, it can be shown that there are no statistically significant differences among the obsered split proportion alues and the estimated alues using either the L E or the LS methods when tested at the 95% leel of confi- 44 JOURNAL OF RANSORAION AND SAISICS V5, N/3 00

10 ABLE Standard Deiation of the LS and the L E Estimators for the eak eriod (7 9 am Absolute Absolute Asymptotic Bootstrap Asymptotic Bootstrap percent percent SD SD SD SD error error OD kj kj SD LS SD LS (AE SD LE SD LE kj kj (AE ( kj ( kj ( ( Key: OD kj Number of ehicles departing from origin node k that arrie at destination node j kj SD SD AE ( LS kj ( LE kj Obsered split proportion matrix between origin node k and destination node j Estimated standard error of split proportion between origin node k and destination node j using the LS estimator Estimated standard error of split proportion between origin node k and destination node j using the L E estimator Difference between bootstrap SD and asymptotic SD diided by bootstrap SD dence his result indicates that either method would be appropriate for this dataset It is important to note, howeer, that the AVI olume data are of ery high quality with a low error rate, which is not the case for inductance loop data he standard deiations of the residuals for columns through 6 are 070, 9, 734, 043, 93, and 6, respectiely, which are relatiely close he residuals iewed using a normal probability plot indicate that the hypothesis that the errors are normal can be accepted Lastly, the off-diagonals of the sample correlation matrix indicated a small correlation between the columns of the residuals he aerage absolute alue of the correlations is 033 Based on these obserations, the assumption that the errors are independent and hae the iid same normal distribution (ie, ε ~ N( 0, σ t I appears alid for this example Checking the assumptions allowed us to apply the input-output model presented in this paper to the Houston AVI data While the AVI example helped alidate the methods deeloped in this paper, it does not illustrate the robustness of the L E relatie to the LS estimator because of the generally high quality of the AVI data herefore, robustness is illustrated using a Monte Carlo simulation study motiated from the AVI test bed example A binomial distribution, with probability parameter H, which represents the proportion of contamination, was used to sample the destination olume matrix across the 7 time periods and 6 destinations wo separate scenarios were examined In the first scenario, the chosen cells are set to zero, which mimics a detector failure In the second scenario, the chosen cells are inflated to mimic chatter in the data he inflated alues were set to GAJEWSKI, RILE, DIXON & SIEGELMAN 45

11 FIGURE Simulation MSE Results for Contaminated Data 30 Mean squared error 5 L E: Scenario 0 5 LS: Scenario 0 LS: Scenario L E: Scenario roportion of contamination 3,40 ehicles per half-hour, which is higher than capacity and consequently the alues are not feasible A Monte Carlo experiment was performed 00 times for each scenario for leels of H ranging from 0 to 080 in increments of 0 he mean squared error (MSE as a function of the probability of destination detector failure for both scenarios is shown in figure It can be seen that as proportion of contamination (H increases, the MSE for both estimators also increases under each scenario Note that for both scenarios the rate of increase in MSE for the L E estimator is much lower than that of the LS estimator For example, when the percentage contamination is 0% the MSE for the L E estimator is approximately 0, which is only 0% and 7% of the MSE for the LS estimator under scenarios and, respectiely It can also be seen that while the L E estimator is more robust than the LS estimator for lower alues of H, after the probability of contamination reaches a certain point both estimators are comparably poor (or approximately flat due to Monte Carlo error he sum of the MSE, which may be considered a surrogate for the ariance, was subsequently calculated using the simulated data and the asymptotic equations using the uncontaminated data (ie, H 0 he sums of the MSE for the LS estimator were 0053 and 0059 for the simulation and asymptotic analyses, respectiely he sums of the MSE for the L E analyses were 0088 and 0090 for the simulation and asymptotic analyses, respectiely hus, the alues deried using the asymptotic theory approximately equal the alues deried from the simulation results and demonstrate the theory deried earlier in this paper o study the effects of errors that are not as seere as those in scenarios and, and consequently harder to detect, a second sensitiity analysis was performed In this case the percentage error was aried between 75% and 75% and, as before, a binomial distribution was used to perturb the destination matrix olume measurements A sensitiity analysis was performed on the percentage contamination, which ranged from 0 to 06 in increments of 0 As in the earlier experiments, the 46 JOURNAL OF RANSORAION AND SAISICS V5, N/3 00

12 FIGURE 3 Efficiency of the L E to the LS ersus ercentage Error of Detector Accuracy Efficiency (MSE L E/MSE LS 6 4 H H H 04 H H 0 H % 50% 5% 5% 50% 75% ercentage error Monte Carlo simulation was carried out 00 times for each simulation he results are presented in figure 3, which shows the ratio of the L E estimator MSE diided by the LS estimator MSE as a function of the percentage of detectors experiencing errors For example, it can be seen that when the detector error rate is 5% and the probability of a gien detector experiencing difficulty is 50% (H 05, the relatie efficiency is 05 hat is, the L E estimator MSE is approximately half of the LS estimator MSE, indicating that the L E estimator is more robust to detect error for this scenario he second point to note about figure 3 is that when there is no data contamination the MSE of the L E estimator is approximately 60% larger than the MSE of the LS estimator It can be seen that as the detector percentage error grows, the efficiency of the L E estimator relatie to LS estimator rises at an increasing rate before it plateaus at approximately an absolute alue of 50% In addition, the choice of when to use the L E estimator is a function of the expected contamination rate and the expected percentage error in detector accuracy As shown in figure 3, wheneer the relatie efficiency dips below 0 the L E estimator would be preferred For this example, when the percentage error of detector accuracy is less than approximately 5% the LS would be chosen Howeer, when the detector percentage error is greater than 5%, it can be seen that the L E estimator would be preferred for all alues of H less than 05 At this point, it should be emphasized that the L E approach is a statistical technique that is not limited to an input-output model but can be generalized to any number of OD problem formulations In the case of this particular problem, other detectors (main lanes with olume recorders that proide additional olume information could be accounted for by substituting the input-output model with a general link olume model where the output is modified to include mainline olumes In this way, the L E estimator would be robust to malfunctions in any of the detectors In addition, confirmatory factor analysis can be used when other detectors in the system fail by imputing the missing data (ark et al 00 he GAJEWSKI, RILE, DIXON & SIEGELMAN 47

13 unknown olumes at each origin are mathematically equialent to the unknown scores in confirmatory factor analysis herefore, when a loop failure occurs, some of the origin olumes are obsered and some are not herefore, confirmatory factor analysis and the models deeloped in this paper can be combined to form a mixed model he obsered destination olumes are contained in matrix D, the obsered origin olumes are in matrix O, the unobsered origin olumes are in matrix O, and the split proportions are placed in matrix he mixed model is thus D O O + ε, where the split proportions are [ ] partitioned For instance, consider the AVI example preiously discussed Suppose the olume counts at origin one or origin six cannot be obsered because of detector failure herefore, origins two through fie correspond to O and origins one and six correspond to O Under the notation described aboe, the least squares objectie function becomes ( Dt [ Ot + Ot] ( t [ t t ] MLS min D O + O Ex, t Next we sole for O and minimize the objectie function with respect to he unknown portion of O is O, thus Ot ( Dt Ot ( herefore the objectie function is MLS min D ( D O ( O t t + t t t D ( ( t Dt O + t O t ( ( min D t I Ot I t D I ( O I ( t t ( min ( Dt O I t t ( Dt O t I ( ( ( arg min D ( t Ot I Dt Ot t Estimates from this objectie function produce a sequence that conerges to the fixed but unknown true parameter (Gajewski 000 Remark: Confirmatory factor analysis occurs when all of the regressors are unknown resulting in being empty he objectie function, similar to confirmatory factor analysis, is arg min O O, where ( I ( Ordinary least squares regression occurs when is empty and t arg min O D O ( Dt t ( t t t t t In most cases, model identifiability of the parameters holds because of the number of nonfeasible OD pairs he biggest challenge is meeting the ( ( assumption where is full rank, assuming k k is the matrix composed of the columns containing the assigned zeros in the kth row with those assigned zeros deleted he nonfeasible OD pairs arise from the fact that a ehicle cannot hae certain origin or destination combinations Details of model identifiability and rank are found in ark et al (00 Note that the LS method is used to demonstrate the approach, because the AVI data were found to be well behaed It should be noted that under the conditions presented in the theory section of this MLS p paper 0 hat is, as goes to infinity the estimator under the mix model using least squares conerges to the true split proportions (Gajewski 000 A test of the imputation technique on data from the test bed was subsequently performed It was assumed that the olume from the first origin and the last origin are missing and the techniques from confirmatory factor analysis were used to impute the missing origin olumes able 3 shows the 48 JOURNAL OF RANSORAION AND SAISICS V5, N/3 00

14 ABLE 3 Estimated AM eak Origin Ramp Volume Using Imputation Methodology Origin Origin Origin 6 Origin 6 obsered estimated ercentage obsered estimated ercentage olume olume error olume olume error NA Continues results, and it can be seen that the aerage percentage error for origins one and six are 30% and 3%, respectiely It can also be seen that the AE was lowest for the origin ramps that had the highest olume he important feature of the aboe technique is that it proides a consistent estimator for the split proportions despite missing a portion of the input information While there are other imputation techniques (Chin et al 999; Gold et al 00, it is not clear if these techniques proide consistent estimators herefore, the aboe algorithm can be used when input information is missing or in situations where the input information is known a priori to be faulty In this latter case, the faulty input olume would be remoed prior to the estimation of the split proportions Remark: he formulation deried in this paper was presented in OD format Alternatiely, in the data imputation format, the origins, O, can be treated as output parameters and the model formulated as O D+ε his is a DO formulation GAJEWSKI, RILE, DIXON & SIEGELMAN 49

15 ABLE 3 Estimated AM eak Origin Ramp Volume Using Imputation Methodology (continued Origin Origin Origin 6 Origin 6 obsered estimated ercentage obsered estimated ercentage olume olume error olume olume error NA Aerage error 30 Aerage error 3 where the split proportion kj represents the proportion of ehicles exiting at k and entering at j he OD matrix was calculated for the full AVI data under both formulations he OD matrix, under the DO formulation, was calculated using the LS estimator he correlation between the OD matrix calculated from the DO model and the OD matrix calculated from the OD model is 0908; the correlation between the OD matrix calculated from the DO model and the true OD matrix is 0954; and the correlation between the OD matrix from the OD model and the true OD matrix is 0993 herefore, the OD matrices from both formulations were approximately equal for this test example CONCLUDING REMARKS Origin-destination matrices are an integral component of off-line traffic models and real-time adanced traffic management centers he recent deployment of IS technology has resulted in the aailability of a large amount of data that can be used to deelop OD estimates Howeer, these data are subject to inaccuracies from a ariety of sources In this situation, the data may be cleaned and then used with existing OD estimators Alternatiely, OD estimators may be deeloped that are robust to the outliers characteristic of IS data his latter approach was the focus of this paper A robust estimator based on the L E theory was first deried and compared with a traditional least 50 JOURNAL OF RANSORAION AND SAISICS V5, N/3 00

16 squares estimator In addition, closed-form asymptotic distributions for the ariance were deried for both estimators his is an important contribution because it allows ariances, and therefore standard errors, to be calculated for the estimates It was shown that while the L E estimator was less efficient than the LS estimator, it was more statistically robust to bad data he models were applied to a corridor on Interstate 0 in Houston, exas he test bed was instrumented with AVI technology and therefore both AVI olumes and split proportions are aailable his test bed proides a unique opportunity for comparing the estimators because an obsered OD matrix is aailable, which is extremely rare for these types of studies he asymptotic ariance estimates were found to be slightly conseratie as compared with the estimates deried using a bootstrap method It was shown that while the LS and the L E estimators had aerage mean absolute error ratios of approximately 50%, both replicated the obsered OD at the 95% leel of confidence In addition, the accuracy of the estimates was highest for the OD pairs with higher relatie olumes A Monte Carlo simulation was carried out to test the robustness of the techniques under different error types and error rates It was found that the L E estimator was much more robust than the LS estimator to outliers Howeer, after a certain threshold, which was 50% on the I-0 test network, the models were found to perform equally poorly Lastly, a method for imputing missing origin data was deeloped and illustrated using the LS estimator It should be noted that the L E approach is not limited to the input-output model used in this paper, but rather can be generalized to any number of OD problem formulations For example, information from other detectors, such as those commonly found on main lanes, could also be added as a dependent ariable and a consistent OD estimator could be formulated In this way, the L E estimator would be robust to malfunctions in any of the detectors In addition, it was assumed in the model deriation that there was no prior information regarding the split proportion matrix he model can also be generalized to incorporate prior information, which would help add structure to the oer-parameterized model and could potentially reduce the percentage error of the estimates REFERENCES Ashok, K 996 Estimation and rediction of ime- Dependent Origin-Destination Flows, hd Dissertation Massachusetts Institute of echnology, Cambridge, MA Basu, A, I Harris, N Hjort, and M Jones 998 Robust and Efficient Estimation by Minimising a Density ower Diergence Biometrika 85(3: Bell, MGH 99 he Real ime Estimation of Origin- Destination Flows in the resence of latoon Dispersion ransportation Research B 5(/3:5 5 Cascetta, E 984 Estimation of rip Matrices from raffic Counts and Surey Data: A Generalized Least Squares Estimator ransportation Research B (4/5:89 99 Cascetta, E and S Nguyen 988 A Unified Framework for Estimating or Updating Origin/Destination Matrices from raffic Counts ransportation Research B (6: Chin, S, DL Greene, J Hopson, H Hwang, and B hompson 999 owards National Indicators of VM and Congestion Based on Real-ime raffic Data aper presented at the 78th Annual Meetings of the ransportation Research Board, Washington, DC Daison, A and D Hinkley 997 Bootstrap Methods and heir Applications New York, NY: Cambridge Uniersity ress Dixon, M 000 Incorporation of Automatic Vehicle Identification Data into the Synthetic OD Estimation rocess, hd hesis exas A&M Uniersity, College Station, X Gajewski, B 000 Robust Multiariate Estimation and Variable Selection in ransportation and Enironmental Engineering, hd hesis exas A&M Uniersity, College Station, X Gold, DL, SM urner, BJ Gajewski, and CH Spiegelman 00 Imputing Missing Values in IS Data Archies for Interals Under Fie Minutes aper presented at the 80th Annual Meetings of the ransportation Research Board, Washington, DC Hampel, F, E Ronchetti, Rousseeuw, and A Werner 986 Robust Statistics: he Approach Based on Influence Functions New York, NY: John Wiley and Sons Huber, 98 Robust Statistics New York, NY: John Wiley and Sons Maher, MJ 983 Inference on rip Matrices from Obserations on Link Volumes: A Bayesian Statistical Approach ransportation Research B 7(6: McNeil, S and C Henderickson 985 A Regression Formulation of the Matrix Estimation roblem ransportation Science 9(3:78 9 GAJEWSKI, RILE, DIXON & SIEGELMAN 5

17 Nihan, NL and MM Hamed 99 Fixed oint Approach to Estimating Freeway Origin-Destination Matrices and the Effect of Erroneous Data on Estimate recision ransportation Research Record 357:8 8 ark, ES, CH Spiegelman, and RC Henry 00 Bilinear Estimation of ollution Source rofiles and Amounts by using Multiariate Receptor Models (with discussions Enironmetrics 3: Robillard, 975 Estimating the OD Matrix from Obsered Link Volumes ransportation Research 9:3 8 Scott, DW 999 arametric Modeling by Minimum L Error, echnical Report 98-3 Department of Statistics, Rice Uniersity, Houston X Sherali, HD, N Arora, and AG Hobeika 997 arameter Optimization Methods for Estimating Dynamic Origin- Destination rip-ables ransportation Research B 3(:4 57 errell, GR 990 he Maximal Smoothing rinciple in Density Estimation Journal of the American Statistical Association 85:470 7 urner, S, L Albert, B Gajewski, and W Eisele 999 Archied IS Data Archiing: reliminary Analyses of San Antonio ransguide Data ransportation Research Record 79:85 93 Van Zuylen, H 980 he Most Likely rip Matrix Estimation from raffic Counts ransportation Research B 6(3:8 93 AENDIX Notation he following notation seres as a reference guide throughout this paper: Number of time periods Each time period is of duration t p q r O D O t Number of destinations in the system Number of origins in the system Number of nonfeasible elements of Matrix of obsered olumes from q origins oer time periods ( by q Matrix of obsered olumes from p destinations oer time periods ( by p Row ector of obsered olumes from q origins from the t th row, t,,3,, D t Row ector of obsered olumes from p destinations from the t th row, t,,3,, O tk Element (t,k of the matrix O, t,,3,,; k,,3,, q Split proportion matrix between q origins and p destinations (q by p kj roportion of ehicles that enter at origin k and exit at destination ramp j where k,,3,,q, and j,,3,,p j Column ector of q split proportions from the j th destination, j,,3,, p ε Matrix of random errors with n rows and p columns A Vectorization of matrix A he operator A takes the columns of A and stacks them on top of each other For example, Stacked ersion of ε ε t Row ector of p elements from the t th row of ε, t,,3,, Stacked ersion of M Matrix that maps the elements of that are nonfeasible to zero M is of size r qp O Stacked ersion of the matrix O Kronecker product A B ( a ij B X * he regressors used in the stacked model, X * ( I O, X * p is p qp G Equality constraint matrix ε G [G G ] artition of G into matrices G and G, where rank(g rank(g q + r, and G is (q + r (q + r G is (q + r (pq q r p Iq G M Note that G is permuted until G is full rank 5 JOURNAL OF RANSORAION AND SAISICS V5, N/3 00

18 G he nonsingular portion of the constraint matrix G G he left oer portion of the constraint matrix G artition of such that the elements correspond to [G G ] he size of and are r + q and pq r q, respectiely he portion of that corresponds to G he portion of that corresponds to G * * * artition of X * such that the elements correspond to [G X [ XX] G ] q Column ector of q ones 0 r Column ector of r zeros g G q g, where g 0r σ I p Variance for each row of ε * Y Y Y XG g (p Y t he t th element of Y * * W W ( X XG G (p (pq q r W t he t th row ector of W Z Z Y r / σ (p Z t he t th element of Z U U W ( p (pq q r σ U t he t th row ector of U W tk he (t,k element of W δ LS L E ψ ( ρ( x N( u,σ ε δ (p σ Estimated ector of using least squares Estimated ector of using L E Influence function (see Hampel et al 986 x ( ( ρ x ψ zdz Normal distribution with mean µ and ariance σ Detail heory Step Condition he split proportions ( are positie for all feasible origin and destination combinations and equal to zero for all nonfeasible combinations herefore the general model is written algebraically as follows: D O + ε, where { 0, and condition} and ε iid t kj ( σ I p ~ 0, j kj he detailed deriation of the L E objectie function is min F [ ( ( ] min f D O f D O dd L E tj t j tj t tj 0 f ( D tj O t 0 dd tj} [ ] ( tj t j tj ( tj t j min f D O dd E f D O { ( A Because the third component in the second line of equation A does not hae the optimization parameter (but only the true parameter 0, it can be taken as a constant and ignored during optimization he empirical density function is used to estimate the expectation in the second line of equation A (Scott 999 Details of the following results appear in Gajewski (000 he error structure for the reduced problem is shown in equation A + p [ ( ] tj t j tj ( tj t j min f D O dd f D O p t j ε ( σ I p ~ 0, (A GAJEWSKI, RILE, DIXON & SIEGELMAN 53

19 herefore, equation A3 proides the estimate of the reduced regression parameters under LS, and it can be seen that it is unbiased he ariance of this estimate is shown in equation A4 Note that equations A3 and A4 are based on the assumption that W is full rank Lemma relates the rank of the origins, O, to that of the rank of W Lemma : If O is full rank, then W is full rank (See Gajewski (000 for proof Note that it is assumed that there will be multiple days worth of data and multiple time periods in each day herefore, the ariation of olume between days and within days causes O to be full rank and therefore W is full rank Similar to the ariance of the estimates in equation A4, the ariance estimates for LS are presented in equations A5 and A6 he coariance between the estimates LS ( LS W W W Y LS (A3 E WW WW ( ( LS ( (( * (( ( Var Var W W W Y Var W W W Y X G g ( WW σ LS ( ( LS ( LS ( Var Var G g G G G Var G G are shown in equation A6 LS ( A4 (A5 LS LS (, ( LS LS, ( GVar ( W W σ and Co Co G g G G LS G ( G G A6 assumed to be in the interior of the parameter space From a physical iewpoint, this means that the true split proportion oer the long term would neer lie on the boundary (ie, be equal to zero for a feasible OD pair It is assumed that this assumption is alid on the highway networks where most IS traffic monitoring deices are located In addition, if this assumption is iolated, the origins and destinations can be combined to alleiate the problem Regardless, this assumption will need to be checked when applying the model he asymptotic distributions for the two estimation methods are deried using properties of M- estimators (Huber 98 Equation A7 shows the L E model As discussed, σ is estimated prior to using equation A7 Let N(A,B be the normal probability density function with a mean of A and ariance B, then arg min FLE p arg min N D, p πσ t j p arg min N( Dtj O t j, σ p t j with A reparameterized model may be obtained by using the ariance to standardize the model Let Z Y and U W σ σ herefore, the reparameterized reduced model will be Z U + δ where δ ~ 0, I p where and the objectie ( function, is shown below: ( tj O t j σ { 0, and condition } min F kj LE j kj p Z min exp i i U i ( A7 ( A8 Notice that the reparameterization causes the problem to be unconstrained except for the nonnegatiity constraints When deriing the asymptotic distributions of the estimators, it is Because the L E is an M-estimator, the distribution properties are straightforward to derie and will be useful in the deriations of the distribution 54 JOURNAL OF RANSORAION AND SAISICS V5, N/3 00