Analyzing Municipal Blockage Failure Datasets for Sewer Systems Yongliang Jin, 1 and Amlan Mukherjee 2

Analyzing Municipal Blockage Failure Datasets for Sewer Systems Yongliang Jin, 1 and Amlan Mukherjee 2 1 Graduate Research Assistant, Department of Civil and Environmental Engineering, Michigan Tech, Houghton, MI 49931; email: yjin@mtu.edu 2 Assistant Professor, Department of Civil and Environmental Engineering, Michigan Tech, Houghton, MI 49931; email: amlan@mtu.edu Abstract: The objective of this research is to analyze historical records of random nonmechanistic failures in civil infrastructure systems. In this paper, we particularly focus on blockage failures in sewer systems that result from combination of external factors, such as variations in usage loads, construction quality and soil characteristics. The study uses historical blockage failure records maintained by municipalities. However, such datasets often tend to be incomplete and/or not representative of failure patterns. We discuss the application of statistical tests specifically the Kolmogorov-Smirnov (KS) test and Anderson-Darling (AD) goodness-of-fit test to estimate (i) if the datasets are complete and representative enough, and (ii) estimate parameters of distributions that appropriately characterize failure event arrival pattern. A dataset consisting of sewer blockage failures for ten years from a small municipality in the State of Michigan was used. The methods discussed in this paper can be used to implement reliability models of other related infrastructure systems as well. Introduction The motivation of this research is to provide decision-makers in state and municipal agencies tools to estimate and budget expenditures for sewer line maintenance. Specifically, the focus is on aiding efficient and cost effective decision-making in municipalities with aging sewer systems. Sewer pipes are often inspected using closed circuit television (CCTV) to apprehend blockages before backups occur (Sinha 2004). Instead of depending on inspection methods, our approach uses historical records of blockage failures maintained by municipalities. Analysis of such historical data can significantly enhance the decision-making process involved in prioritizing maintenance decisions. We illustrate the application of this method using sewer line blockage failure data from a small municipality in the State of Michigan. Background 597

598 CONSTRUCTION RESEARCH CONGRESS 2010 Literature pertaining to serviceability predictions of sewer infrastructure systems can be classified using the following approaches to service assessment: (i) By studying mechanistic system failures using hydraulic properties of sewer lines and pumping (Bennis et al. 2003; Ermolin et al. 2002), (ii) By studying system failures as random events that can be a direct or indirect outcome of mechanistic and non-mechanistic events. The methods used in this approach are: a. Use historical failure data from case studies to predict behavior of specific systems. Typically, specific random failure events such as sewage pump failure and sanitary sewer overflows are modeled using historical data (Korving et al. 2006; Sumer et al. 2007). b. Assume the Markov property to develop general models that assess the condition of infrastructure for modeling pipe condition and deterioration, and optimizing maintenance schedules. (Abraham et al. 1998; Chughtai and Zayed 2008; Micevski et al. 2002; Wirahadikusumah et al. 2001). The research presented in this paper specifically falls in category (ii.a). It analyzes sewer system failures resulting from blockages using historical failure data for a small municipality. Stochastic methods are used in assessing failure of infrastructure systems due to discrete events, such as sewer overflows and sewer blockages that are modeled as random events. By definition, a stochastic process X={X(t), t T} is a collection of random variables X(t), which denote the state of a process at time t (Ross 1993). When time is measured discretely, X(t) is either true or false based on whether an event occurs or not at the time point t. When time is continuous, the inter-arrival times between failure events are recorded. A continuous stochastic process in which events occur independently of one another with no overlapping arrivals at a constant rate reduces to a Poisson process (Kingman 1993). Then the inter-arrival times are exponentially distributed. In other areas of Civil Engineering, similar approaches have been used in modeling rainfall (Cowpertwait et al. 2007) and analyzing motor vehicle crash frequencies (Ma and Goulias 2007). In such studies, the challenge has been in appropriately estimating parameters that define the processes. Indeed, the attractive memoryless property, unique to the exponential distribution (and its discrete counterpart the geometric distribution) provides significant analytical advantages (Vaughan 2008). While the assumption may be justified in certain cases, often in the absence of validation, it leads to conservative estimates of reliability (Chu and Durango-Cohen 2008). In reality, failure rates often are not constant. The Weibull and Gamma family of distributions provide alternative better fits in such cases. The exponential distribution is a special case of both these families of distributions. Specifically, a 2- parameter Weibull distribution is a continuous probability distribution with the density function: k t k k 1 ( t/ α ) f (; t α, k) = ( ) e t 0 α α and cumulative distribution function:

CONSTRUCTION RESEARCH CONGRESS 2010 599 k (/ t α ) Ft (; α, k) = 1 e t 0 where k is the shape parameter, is the scale parameter. When k=1, the Weibull distribution has constant failure rate at any time t and reduces to the exponential distribution. The 3-parameter Weibull introduces an additional location parameter on top of two-parameter Weibull to describe a right shift of the distribution. The density function is given by: k t τ k k 1 (( t τ)/ α) f (; t α, k) = ( ) e t τ α α and distribution function is k (( t τ)/ α) F(; t α, k) = 1 e t τ Our research efforts in this study is similar to Korving s (2006) work as both aim to evaluate reliabilities of sewer systems through analysis of failure data. Instead of focusing on pump failure data, sewer blockage dataset is the target. This study also supports the use of interactive simulations to support decision making in codependent infrastructure systems (Mukherjee et al. 2010). Methodology The main challenges in characterizing failure datasets using stochastic models are 1) datasets could be incomplete or not representative, 2) finding suitable distributions to describe the event arrival process. While the Poisson process is often assumed, other distributions might provide better fits and provide deeper insight into the behavior of the system. These challenges were addressed by analyzing a sewer blockage dataset. The municipality under observation is a small town with no major industry located in the vicinity. The predominant use is housing while commercial use only accounts for a small portion. The town consumes about 1.1 million gallons of water per day. There are about 1500 customers, of which 1100 are single-family units (Houghton-City 2009). Sewer event data was obtained from hand written records and manually organized in digital spreadsheets. The dataset covers sewer events collected on all sewer pipelines running through the town. Events recorded were categorized into new construction, maintenance activities, failures and blockage rescues. Sewer blockage event information included occurrence date and location, type of obstruction, and total time required to repair the problem. The entire dataset consisted of records from 1996 to 2006. The study used records from 1996 to 2003, as the records between 2004 and 2006 were ill maintained and incomplete. Most blockage events reflected in the historic data records are related to pipes with sizes smaller than 24 inches in diameter. No repeated blockages were found at one specific location. Data analyses were performed on the inter-arrival times datasets derived from blockage events. The analysis procedure is described in Figure 1. Raw failure data was evaluated and blockage failures were identified by occurrences of sewage backup events or overflow events. In order to ensure that the dataset provided a representative sample, a two sample Kolmogorov-Smirnov test was done to check if sub-sets of the data were similar to each other and therefore by extension, if the dataset was a suitable sample for study. This was followed by a goodness of fit analysis using the Anderson-Darling test to select a distribution that suitably

600 CONSTRUCTION RESEARCH CONGRESS 2010

H a : The two datasets do not have a common distribution. Figure 2(a) below illustrates comparisons between the sub datasets and the applied dataset, plotted on a cumulative fraction basis. Under a given 95% confidence level, the null hypothesis cannot be rejected (p=0.8004). Figures 2(b)-(d) display comparisons between samples from data split1, data split2 and applied dataset. The hypothesis of a common distribution can be accepted for data split1 and data split2. Similar conclusions can be made regarding the split datasets and applied dataset as illustrated in Figures 2(c) and 2(d). Having confirmed that the dataset has a common distribution the next step was to establish specific distribution trends to characterize blockage arrivals. F(x) F(x) 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 Empirical CDF AppliedData EntireData 0 0 50 100 150 200 250 x 1 CONSTRUCTION RESEARCH CONGRESS 2010 601 (a) Empirical CDF Split1 Split2 0 0 10 20 30 40 50 60 70 80 90 x 0 0 10 20 30 40 50 60 70 80 90 x (c) (d) Figure 2: KS-test Comparison Fraction Plot between: (a) Applied dataset and Entire dataset, D=0.0532, P=0.8004; (b) data Split1 and Split2, D=0.0976, P=0.4799; (c) Split1 and Applied dataset, D=0.0503, P=0.9697; (d) Split2 and Applied dataset D=0.0472, P=0.9796. Distribution Fitting of Inter-event Times This section describes how the distributions of inter-event times were determined. The Anderson-Darling (AD) goodness-of-fit test was used. This null and alternate hypothesis for this test is as follows: H 0 : The dataset follow a specific distribution, H a : The dataset does not follow a specific distribution, Significance level: 0.05. Tests were performed upon datasets based on a yearly duration as well as datasets combined per wider time ranges, namely per 2 years, per 4 years and per the entire dataset. Test results were compared and discussed respectively for each timing combination. Exponential distribution fitting and analysis: Table 1 lists exponential fitting results applied upon categorized datasets. Given a 95% confidence level, the F(x) F(x) 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 Empirical CDF (b) Empirical CDF split1 AppliedData Split2 AppliedData 0 0 10 20 30 40 50 60 70 80 90 x

602 CONSTRUCTION RESEARCH CONGRESS 2010 null hypothesis of an exponential distribution could not be rejected for each annual dataset. When the period was expanded to 2 years or higher, the null hypothesis could be rejected at the given 95% confidence level. There are multiple factors leading to such test failures. Major external weather events, and/or variations in external forces may induce high frequency events may be one reason. To further explore this, a Weibull distribution was applied upon the inter-arrival times datasets. Table 1: Exponential distribution tests results for all timing combinations (1year, 2 years, 4years and entire dataset) Year Failure rate P value 1996 12.57 0.0796 0.138 1997 8.332 0.1200 0.664 1998 12.04 0.0831 0.052 1999 7.989 0.1252 0.109 2000 10.72 0.0933 0.204 2001 7.831 0.1277 0.245 2002 13.19 0.0758 0.589 2003 9.446 0.1059 0.084 96-97 10.17 0.0983 0.018 97-98 10.16 0.0984 0.011 98-99 9.930 0.1007 0.005 99-00 9.310 0.1074 0.024 00-01 9.147 0.1093 0.041 01-02 10.37 0.0964 0.039 02-03 10.92 0.0916 0.045 96-99 10.39 0.0962 <0.003 00-03 10.19 0.0981 <0.003 96-03 10.28 0.0973 <0.003 Weibull distribution fitting and analysis: Multiple factors contribute to blockage occurrences. As a result, failure arrivals are not completely random, as a Poisson process would imply. Hence, the Weibull distribution was tested for a suitable fit. Table 2 lists Weibull test results, it can be seen that except data sub-sets 2000-2003 and 1996-2003, all others generate p values greater than 0.05, which support a Weibull distribution fit for the inter-arrival times. Further examination on the tests reveals that the shape parameters are smaller than 1 for all datasets. This means that compared to an exponential distribution, a higher probability density was noticed for short inter-arrival times. Given that a Weibull distribution reduces to an Exponential distribution when the shape parameter equals to 1, it can be seen from the shape parameters in Table 2, that the behavior of the dataset can be approximated using a Poisson process over short observation periods. However, when the observation period is extended, various impacts skew this exponential trend. Thus, a Poisson process cannot be validated for the entire observation period. Instead, blockage arrivals are better characterized by Weibull distributed inter-arrival times.

CONSTRUCTION RESEARCH CONGRESS 2010 603 In order to understand why Weibull is a better fit, we further investigated the underlying contexts. Such Weibull inter-event times are mainly caused by more frequent than a general random arrival of blockages during certain times of the year. To illustrate this, figure 3 below depicts total blockage numbers versus each month categorized from the entire data population. High numbers of blockages were noticed from March to May. This can be attributed to increased flow from snowmelt. Another gentler increase was also observed at the end of year, which is believed to be caused by intensive human activity. Results in Table 2 also reveal that the dataset fails the null hypothesis of a Weibull distribution for dataset 00-03 and the entire dataset respectively. To handle this situation, a 3-parameter Weibull distribution was further applied, where a new location parameter is introduced. It can be seen from Table 2 that all datasets tests have p values greater than 0.05. These additional location parameter adjustments assist to handle the heavy tails accumulated over time, although they do not significantly impact parameters for distribution shift. Analysis of inter-arrival times datasets was also performed by using the Gamma distribution family. Test results were found to be similar to those from Weibull distribution tests, both of which are two- or three- parameter distributions. Weibull distributions provide better fits for datasets covering wide time periods. Figure 3: Number of blockages versus month Table 2: Weibull distribution tests results for all timing combinations Shape Year parameter Scale P value 1996 0.7990 11.11 >0.25 1997 0.8868 7.490 >0.25 1998 0.8101 10.73 >0.25 1999 0.8180 7.144 >0.25 2000 0.8438 9.800 >0.25 2001 0.8836 7.318 >0.25 2002 0.9435 12.85 >0.25 2003 0.8297 8.487 >0.25 96-97 0.8201 9.096 >0.25 97-98 0.8192 9.085 >0.25 98-99 0.8052 8.795 >0.25

604 CONSTRUCTION RESEARCH CONGRESS 2010 99-00 0.8305 8.418 >0.25 00-01 0.8666 8.468 >0.25 01-02 0.8720 9.645 0.21 02-03 0.8702 10.15 0.231 96-99 0.8042 9.178 >0.25 00-03 0.8612 9.042 0.037 96-03 0.8351 9.308 0.012 Table 3: 3-parameter Weibull tests for datasets Year Shape parameter Scale Location parameter P value 96-99 0.7817 8.965 0.05938 >0.50 00-03 0.8343 9.154 0.08414 0.105 96-03 0.8102 9.079 0.07006 0.061 To sum up statistical analyses in this section, Figure 4 depicts all test results upon available inter-arrival times datasets. Specifically, a one parameter exponential distribution fits annual datasets favorably. Standard Weibull (2-parameter) distributions generate better fits for datasets covering 2 years ranges. Similar results are indicated from Gamma distribution test fits. The exponential distribution fit can only be approximated for short observation periods. A Weibull distribution fit or close approximations were also noticed for 4-year datasets and the entire dataset, an additional location parameter assists to produce better fits at this level. Theoretically, an exponential distribution of inter-arrival times implies an underlying Poisson arrival process. However, multiple real life impacts usually skew this exponential trend to a certain extent. Specifically in this study, snowmelt and intensive human activity lead to Weibull inter-arrival times of blockage event sequences. Figure 4: Test results for different time duration combinations (1year, 2years, 4years, 8years) Note: only successful tests are shown in the chart. w/g/ =all Weibull, Gamma and exponential distributions pass tests, Weibull is the best fit; w/g= both Weibull and Gamma distributions fulfills tests, approximate exponential distributions are noticed; 3-p-w=3 parameter Weibull distribution pass AD tests, which are only used on datasets 4years and entire dataset, approximate 2 parameter Weibull is noticed. Discussion Three target distributions were tested, namely, the one-parameter exponential distribution, the two- and three-parameter Weibull distribution and the Gamma distribution. Usually, a two- or three- parameter distribution is always more likely to fit a data set as additional parameters provide a more complete description of data

sets characteristics. In this study, the shape parameter in the Weibull distribution provided insight into the nature of blockage failures. In addition, a detailed analysis over sub-data sets and consideration of time trends, illustrated the possible influence of local factors and usage patterns. Trend of inter-arrival times in this study displayed a Weibull distribution with a shape parameter k < 1, a close approximation to an exponential distribution (k = 1) as shown in Figure 5. This implied a higher density of shorter inter-arrival times, that identified higher frequency of event occurrences due to snowmelt and periods of increased human activities. Data sets used in this study were collected from a sewer system of a small town. For a large sewer infrastructure system, the same method can be extended and used across data subsets classified by usage areas, blockage scenarios, and soil conditions to get a better understanding of underlying system behavior. Density 0.12 0.10 0.08 0.06 0.04 0.02 0.00 0 10 Distribution Plot Weibull, Scale=10, Thresh=0 20 Figure 5: Probability density function for shape parameter k=1 and k=0.8 Conclusion CONSTRUCTION RESEARCH CONGRESS 2010 605 This paper proposes a methodology to study sewer systems serviceability given a blockage occurrence dataset. Blockage events were found to arrive in a process with Weibull inter-arrival times. Specifically, a Poisson arrival process holds true over short observation periods and an approximate Poisson model can be assumed over longer observation periods. However, a more precise description of Weibull interarrival times was noticed resulting from real life impacts that vary regionally. Further applications can be performed by employing properties of a Poisson process and distribution in order to estimate failure rate in order to assist stalk-holders in the decision making process. References Abraham, D. M., Wirahadikusumah, R., Short, T. J., and Shahbahrami, S. (1998). "Optimization Modeling for Sewer Network Management." Journal of Construction Engineering and Management, 124(5), 402-410. Bennis, S., Bengassem, J., and Lamarre, P. (2003). "Hydraulic performance index of a sewer network." Journal of Hydraulic Engineering, 129(7), 504-510. Chu, C.-Y., and Durango-Cohen, P. L. (2008). "Empirical Comparison of Statistical Pavement Performance Models." Journal of Infrastructure Systems, 14(2), 138-149. 30 X 40 50 60 Shape 1 0.8

606 CONSTRUCTION RESEARCH CONGRESS 2010 Chughtai, F., and Zayed, T. (2008). "Infrastructure Condition Prediction Models for Sustainable Sewer Pipelines." Journal of Performance of Constructed Facilities, 22(5), 333-341. Cowpertwait, P., Isham, V., and Onof, C. (2007). "Point process models of rainfall: developments for fine-scale structure." Proceedings of the Royal Society, 463, 2569-2587. Ermolin, Y. A., Zats, L. I., and Kajisa, T. (2002). "Hydraulic reliability index for sewage pumping stations." UrbanWater, 4, 301-306. Houghton-City. (2009). "2008 Water Quality Consumer Confidence Report." Retrieved 10/10, 2009, from http://www.cityofhoughton.com. Kingman, J. F. C. (1993). Poisson Process, Oxford University Press Inc., New York. Korving, H., Clemens, F. H. L. R., and Noortwijk, M. v. (2006). "Statistical Modeling of the Serviceability of Sewage Pumps." Journal of Hydraulic Engineering, 132(10), 1076-1085. Ma, J., and Goulias, K. G. (2007). "Application of Poisson Regression Models to Activity Frequency Analysis and Prediction." Transportation Research Record: Journal of the Transportation Research Board, 86-94. Micevski, T., Kuczera, G., and Coombes, P. (2002). "Markov Model for Storm Water Pipe Deterioration." Journal of Infrastructure Systems, 8(2), 49-56. Mukherjee, A., Johnson, D., Jin, Y., and Kieckhafer, R. (2010). "Using Situational Simulations to Support Decision Making in Co-dependent Infrastructure Systems." International Journal of Critical Infrastructures, 6(1), 52-72. Ross, S. M. (1993). Introduction to probability models, Academic Press, San Diego, California. Sinha, S. K. (2004). "Intelligent System for Condition Monitoring of Underground Pipelines." Computer-Aided Civil and Infrastructure Engineering, 19, 42-53. Sumer, D., Gonzalez, J., and Lansey, K. (2007). "Real-Time Detection of Sanitary Sewer Overflows Using Neural Networks and Time Series Analysis." Journal of Environmental Engineering, 133(4), 353-363. Vaughan, T. S. (2008). "In search of the memoryless property." Proceedings of the 2008 Winter Simulation Conference, 2572-2576. Wirahadikusumah, R., Abraham, D., and Iseley, T. (2001). "Challenging Issues in Modeling Deterioration of Combined Sewers." Journal of Infrastructure Systems, 7(2), 77-84.