Impact of Lightning Strikes on National Airspace System (NAS) Outages

Transcription

1 Impact of Ligtning Strikes on National Airspace System (NAS) Outages A Statistical Approac Aurélien Vidal University of California at Berkeley NEXTOR Berkeley, CA, USA aurelien.vidal@berkeley.edu Jasenka Rakas University of California at Berkeley NEXTOR Berkeley, CA, USA jrakas@berkeley.edu Abstract Altoug it is reasonably well accepted tat ligtning strikes are a significant cause of outages on te National Airspace System (NAS), tere remains a serious lack of compreensive analyses providing sound estimations of convective weater impact. Te results and metods available generally cover specific outages and try to determine teir causes by compreensively analyzing te ligtning strikes tat occurred in te vicinity of te system. Suc metods are inappropriate wen trying to evaluate te global impact of convective weater on very large systems suc as NAS, wic is composed of more tan 70,000 systems. In tis paper, a statistical metod is developed to estimate te number of outages caused by ligtning strikes, wic is robust on bot te time detection of te outage and te localization of te strikes. In addition, we present results of its application on te NAS outages between 1999 and A preliminary analysis of te problem was carried out in an attempt to establis an initial relation between ligtning strikes and outages. All strikes tat appened in te continental US between 1999 and 2005 were attributed to eac of te 20 Air Route Traffic Control Centers (ARTCC) and to eac mont. Ten, values were plotted against te number of outages occurring in te respective ARTCC to determine if a general pattern exists (Figure 1). Altoug a small positive correlation is noticeable, results are unusable for furter applications as in many cases more ligtning strikes do not imply more outages. Indeed, more advanced tools based on te caracteristics of ligtning strikes are required to explain te outage process. Index Terms - component; ligtning strikes; outages; NAS; NLDN; logit model I. INTRODUCTION Te safety and efficiency of air transportation witin te United States (US) largely relies on te reliability of te National Airspace System (NAS). Ligtning strikes are believed to be one of te major causes of electrical power interruption and occur during critical weater conditions [1] for airborne aircraft. Te metod presented ere is based on te National Ligtning Detection Network (NLDN) observations tat represent more tan 95% of te ligtning strikes in te continental US [2] and provide te time of te strike, its location and te intensity. Te data base contains te location of te first stroke of eac ligtning strike wic can cause errors of until 10 km, based on te algoritm used by te NLDN [3]. Te information concerning te outages, provided by te Federal Aviation Administration (FAA) was gatered manually by te tecnicians in carge of te maintenance of te NAS, utilizing a system of Cause Codes to classify te different categories of outages. Figure 1. Plot of te number of ligtning strikes during a mont and inside a given ARTCC against te number of outages inside te same region and over te same period of time II. CREATION OF THE DATA BASE A. Data Management Te data used for tis study contain all te ligtning strikes in te continental US between Marc 1999 and November 2005 (recorded by te NLDN). Te data contain bot cloud-tocloud (CC) and cloud-to-ground (CG) ligtning strikes. Beside interferences, te former do not impact te NAS systems significantly and are terefore ignored in te study. For eac strike, te data set provides te location, te time and te 1

2 magnitude (in kilo amperes) but does not give te number or te location of eac stroke. Te list of all te outages tat concerned te NAS during te same period constitutes te second data set. For eac outage, te information available is: te type of system, its location, te beginning and end of te outage, and reported cause of te failure. Te Cause Code (i.e. code) corresponding to Convective Weater is 85-3 (Weater Effect Ligtning Strikes) and all outages wit a different code are not used to develop te model. It is interesting to note tat te time of te beginning of te outages is te time wen te outage was detected and not necessarily te time wen it occurred. Finally, as te NLDN covers only te continental US, all te outages concerning systems located in te ARTCC of Honolulu (ZHN), Ancorage (ZAN) and not on te continent (SCT) were also removed. Te number of remaining outages is close to 900. Cose a ligtning strike Distance X NM? YES YES Time Y ours? Correlated Strikes NO NO IGNORED Is te system operational? YES Non-Correlated Strikes Figure 2. Algoritm applied to eac outage and for all ligtning strikes NO B. Correlated / Non-Correlated Strikes Te input of te model is built wit 2 inct selections of ligtning strikes: one group tat caused outages (Target Strikes) and anoter tat occurred in te vicinity of te systems but tat did not cause any outages (Non-Correlated Strikes). Te former cannot be directly created considering te lack of precision of te NLDN data and terefore a larger (and simpler to define) group of strikes tat migt ave caused an outage is required (Correlated strikes): Correlated Strikes: strikes tat migt be responsible for te outage, Target Strikes: strikes tat are considered as responsible for te outage, Non-Correlated Strikes: strikes tat did not cause any outage, Undetermined strikes: strikes tat do not belong to te two previous categories. Te Undetermined strikes are mainly strikes tat occurred in te vicinity of te system but wen it was not operating for any reason (maintenance, outage, etc). Strikes are first selected based on a spatial criteria of X nautical miles and ten attributed to te different groups according to a temporal criteria (Figure 2). C. Time and Space Window Te separation between te different groups of strikes is based on a time criterion and a space criterion. Te name used for te study is Time and Space Window and corresponds to an X NM radius circle around te system and a period of Y ours before te outage. Te final values for X and Y are respectively 5 NM and 48 ours, wic resulted from te analysis of te Cause Code 85-3 outages. An adequate Time and Space Window must give bot a ig level of correlation as we expect tat most of te Cause Code 85-3 outages are caused by ligtning strikes and a small number of correlated ligtning strikes to facilitate te target strikes extraction. Te level of correlation represents te ratio of te number of correlated events (outages) for a period over te total number of events (outages) for te same period. An outage is Table 1: Level of Correlation as a function of te Time Window size and te Space Window size Time 5min 30min % 1% 2% 3% 4% 5% 5% 5% 5% 5% % 8% 12% 17% 20% 23% 25% 25% 25% 25% 0.5 6% 16% 22% 32% 37% 42% 46% 46% 47% 47% % 22% 28% 41% 46% 52% 56% 57% 58% 59% Distance (NM) 1 13% 24% 31% 44% 50% 57% 62% 64% 64% 65% 2 20% 32% 40% 53% 60% 67% 72% 74% 75% 76% 3 25% 35% 42% 57% 62% 70% 76% 78% 80% 81% 4 28% 38% 44% 58% 65% 72% 79% 82% 83% 84% 5 30% 40% 46% 61% 66% 74% 80% 83% 84% 85% 10 31% 41% 48% 63% 69% 78% 85% 87% 88% 89% 15 32% 42% 49% 64% 70% 79% 87% 89% 90% 91% 2

3 Figure 4: Comparaison between clusters tat can overlap and clusters witout a step value considered correlated if its Time and Space Window contains at least one ligtning strike. As te Time and Space window becomes larger, more irrelevant strikes are correlated. Te definition of te window must take into account tis tradeoff between te Correlation Ratio and te relevance of te correlated strikes. Te Time Window represents te maximum period before an outage during wic a strike is considered as a potential cause of outage. Altoug te outage mecanism is not perfectly understood, it is well accepted tat te time between a strike on te system and te failure will be generally very sort. Associated failures are typically caused by over intensity resulting in excessive eat inside te system and tus occur almost instantaneously after te impact [3]. As te ligtning data and outages data ave respectively an average time precision of 5 µs and 1 minute, a time window of a few minutes sould capture most of te cases [3]. Te main issue comes from te detection time of te outages. Systems equipped wit automatic outage reports are automatically considered as down as soon as te outage occurs. For oter pieces of equipments suc as radars, wic are closely monitored, outages are also very likely to be detected witin a few minutes. However, for oter systems, suc as Precision Approac Pat Indicator (PAPI), outages can be only detected in te case were someone tries to use te system or during a routine ceck. Unfortunately, no studies ave been carried on to determine te average detection time of outages depending Figure 3. Spatial representation of te correlated ligtning strikes. Eac dot represents a strike tat occurred witin 5NM of te system indicated on te orizontal axis. Te time between te ligtning strike and te outage of te concerned system is on te vertical axis? 3

4 on te type of systems and teir location. Different simulations performed wit Time windows between 1 minute and 5 days proved tat te detection of an outage can take up to several days altoug more tan 50% of te correlated outages ave at least one ligtning strike in te our prior to te failure (Table 1). Te Space Window represents te size of te vicinity in wic strikes are considered as a potential cause of outage. Its size is te radius of te circle and is centered on te system. In tis case, te incertitude mainly comes from te NLDN stroke gatering algoritm [3]. Different simulations wit space windows between 0.1NM and 15NM sowed a similar beavior compared to te time window. All curves tend to a limit value after 10 NM. Once again, altoug some outages ave teir closest correlated ligtning strike located at more tan 10 NM, more tan 50% of te correlated outages ave at least one ligtning strike witin 0.5 NM (Table 1). Te final size of te Time and Space Window directly determines te correlated strikes, among wic te target strikes, input of te statistical model, will be selected. Its definition is based on a tradeoff between te number of observation tat sould be as large as possible and te relevance of te input. Eac time te Time and Space Window increases, new observations are added but te probability tat tey are actually involved in te outage process decreases. As sowed in Table 1, te correlation ratio reaces a limit in bot cases. Tose values are considered as te maximum number of observations wic can be used and it is decided to use only 95% of tem inside te model. Te last 5% of correlation implies a multiplicative factor of 2 to 3 for te time and space parameters and adds a large number of correlated strikes wic relevance is very low. D. Strike Definition vs Cluster Definition Strikes are defined as te strikes tat caused te outage as opposed to te correlated strikes, wic are te strikes tat migt ave caused te outage. However, tis definition results in many possible solutions as no general rule or model exists to determine, among a group of strikes, wic one actually caused te outage. Clusters were created to solve te problem of best strikes selection. Te idea is to gater correlated strikes over a period of time and ten consider te wole cluster as te outage cause. Tis principle was born from te observation of te time localization pattern of Correlated Strikes tat tend to appear in groups of ig time density (Figure 3). Eac dot represents a correlated ligtning strike and is positioned on te grap depending on its related outage (orizontal axis) and time between te strike and te outage reported time (vertical axis). Because all te represented strikes belong to te Correlated Strikes group, tey are all located witin a range of 5NM from te system. As ligtning strikes appen during tunderstorms, multiple strikes generally it te vicinity of te system witin a sort period. Te goal of clusters models is to isolate tose laps of time of ig convective activity and consider tem as te cause of te outage. Clusters are fully defined by 2 different caracteristics: Maximum Lengt: teir maximum duration in time. Clusters wit maximum lengts between 5 minutes and 48 ours were simulated to find te most significant values. Te lengt of te cluster is defined by te last strike inside te time interval, tus most clusters are sorter tan teir maximum value. Te maximum lengt is set to 48 ours to keep an even comparison between Correlated and Non-Correlated Clusters as correlated strikes are defined for a period of 48 ours. A larger lengt would add strikes inside te Non- Correlated Clusters defined over te 7 years of te study but not to te Correlated-Clusters. Step: te minimum time between te beginnings of two consecutive clusters from a same outage. For simulations, clusters work as time windows wic are slid from te step value until anoter strike is found. For eac outage, te first cluster starts wit te last correlated strike (te closest in time from te outage reported time), and is ten moved from te step value in te past and keeps sliding in te past until it finds a new strike. Te process is repeated until all te strikes ave been attributed to at least one cluster. Te step value s purpose is to prevent te cluster window from missing a group of strikes by cutting it in two different parts (clusters). It is possible to ignore te step parameter by coosing its value equal to te maximum lengt. In tis case, clusters are never overlapping eac oter. Because all strikes ave to belong to at least one cluster, steps values are always smaller or equal tan te maximum lengt of te cluster, even if tis causes some strikes to belong to more tan one cluster. Te step value sould not be too small to limit te number of appearance of te same strikes inside different clusters wic can cause correlations issues wit te statistical model. Tus, te typical steps values are te maximum lengt multiplied by 1/3, 1/2 and 1. It appears tat steps values ave very little impact on statistical model parameters and levels of goodness altoug steps of alf te maximum lengt tend to give better results in tis case. In addition, 1/2- steps clusters limit te multiple occurrences of strikes to two. Terefore, for te rest of te study, all clusters used a step value equal to te alf of teir maximum lengt. III. A. Sape of te model STATISTICAL MODEL Te statistical model used to process te input is te binary logit model: logit (p i ) = log(p i /(1 p i )) = k + β 1 X 1 n + + β n X i n Tis model is used because of te nature of te input tat uses a time criteria to separate te Correlated and te Non- Correlated strikes. Wen te model is applied, it is limited to te clusters located inside te Time and Space Window. As a result, te sum of te output corresponds to a partial sum and 4

5 does not represent te real estimation of te number of Confirmed Outages. For tis reason, an extra tool is required for te analysis and te tool tat seems to work te best is te tresold metod proposed ere, followed later by an aggregation of te models. Te Decision Tresold (DT) of a model is a fixed value above wic an observation is considered as a 1. In oter terms, te model gives for eac cluster a probability tat tis cluster actually caused an outage. If tis value is above te DT of te model, te cluster is considered as a 1 (caused an outage). Te DT of eac model is set to maximize te inction between correlated and non correlated strikes i.e., te proportion of rigt answers given by te model on te input data. Te model is correct if te predicted status of te cluster is te same as te real one. Te 4 different scenarios, represented by letters from A to D (Table 2), are: 0-0 (A): Non-correlated strike considered as not a cause of outage by te model Correct answer 0-1 (B): Non-correlated strike considered as a cause of outage by te model Wrong answer 1-0 (C): Correlated strike considered as a not a cause of outage by te model Wrong answer 1-1 (D): Correlated strike considered as a cause of outage by te model Good answer Table 2: Definition of te goodness of te model Prediction of te model A % B % Real Value 1 C % D % For eac model, te DT is adjusted in order to maximize te value of te goodness. B. Selection of Parameters In tis study, te selection process for parameters is not based on te complex analysis of failure mecanisms. Tere is extensive literature presenting advanced electromagnetic models in wic te goals are to precisely predict te propagation process based on observations made by sensors located near or on te systems. Suc a level of precision is not realistic because of te lack of precision of te data available but fortunately, it is also not needed given te statistical point of view of te study. Our approac is more intuitive because it consists of parameters tat seem more relevant to te model and ave better statistical significance. Te first step of te process is te creation and selection of potentially meaningful metrics. Parameters suc as te intensity of te strike or te ance between te strike and te system undoubtedly ave an impact on te outage mecanism. In addition, te type of system of te area on wic it is installed may also play a role. We also developed Hybrid metrics to increase te versatility and precision. Given tat te maximum expected precision of te NLDN is 500 meters, all values of ances smaller tan tat were modified to be equal to 500 meters. Te second step is te elimination of non-statistically relevant parameters. First, tey ave to matc te intuitive positive or negative effect tey ave on te model. For example, intensity sould ave a positive impact (positive sign for β intensity) meaning tat te iger te intensity, te iger te cance for an outage. Ten, te significance, represented by te p-value, designates te statistically sound metrics. However, because of important correlation issues between te parameters, te p-value can strongly fluctuate from one model to anoter for a given parameter. Type of equipment Duration of te Cluster Number of Strikes ARTCC Mont / Season Time Density Table 3: Rejected Parameters Higest Intensity/Distance² Higest Intensity²/Distance² Sum of Intensity/Distance² Sum of Intensity²/Distance² Duration of te outage ARTCC parameters are owever a special case. Altoug it appears natural to take into consideration te region were te outage occurred, te definition of te Correlated Strikes already covers tis part. Regions wit a larger convective activity will see more ligtning strikes in teir Time & Space Windows. In addition, te caracteristics of te strikes relative to a special region (violent sort storms for example) are also covered by te caracteristics of te strikes. As a result, taking into account te ARTCC would result in a redundancy, wic is wy tey are not used in te models. Te equipment types also required a special treatment as tey generally result in ig p- values, mainly because of te small number of observations for certain types of systems. Large p-values are not acceptable in terms of consistency but te gatering of some equipment types into a larger group solved tis problem. To determine tis new larger group called ZZZ, te outputs of simulations obtained wit different cluster lengts were compared and all non-significant equipment types were moved into tis group. Te ZZZ group is considered neutral and serves as a basis for te value of te parameters of te oter groups and terefore do not appear in te models output (te estimate value is 0). Parameter Name H Int over H Int H Dist S Int over New Type1 Table 4: Description of te final parameters Full Name Higest Intensity over Distance Higest Intensity Smallest Distance Sum of Intensity over Distance Modified type of equipment Description Higest value of Intensity/Distance observed among te strikes of te cluster Higest value of Intensity observed among te strikes of te cluster Smallest value of Distance observed among te strikes of te cluster Sum of all te values of Intensity/Distance of te strikes contained inside te cluster Type of equipment tat underwent te outage. Some equipment types are gatered inside te ZZZ group 5

6 Table 5: Caracteristics of te models depending on te cluster lengt Goodness DT Confirmed 48 ours 40% ours 43% ours 46% ours 47% ours 49% our 50% min 53% min 53% Parameter 5 min 53% Table 6: Statistical Model (1our-long clusters) Estimate Standard Error Wald Ci Square Pr>CiSq Intercept <.0001 H Int over <.0001 H Int H Dist <.0001 S Int over <.0001 Te 9 different models are ten applied to outages codes different tan Cause Code First, te Time & Space Windows are created for eac outage and ten clusters of te different lengts are defined and te corresponding models (wit teir corresponding codes) are ten applied. If, for a given outage and a given cluster lengt, te models return a value iger tan te DT, te outage is considered as a Confirmed Outage. Te total number of Confirmed Outages is fairly constant among te different models, but te results for a given outage migt vary. To consolidate te results, a matrix containing all te outages and te corresponding output for eac model is created (Table 9). Ten, depending on te desired level of conservativeness, it is possible to select a new tresold above wic an outage will be ultimately considered as caused by convective weater. Te tresold is generally cosen as at least te majority of te models, wic corresponds to 5 in our case. Tus, in tis example, only outage 2 is considered as caused by convective weater. IV. CASE STUDY Te model as been developed using 987 outages reported as caused by ligtning strikes (Cause Code 85-3) between 1999 and Wile building te Time and Space Windows, it appeared tat 200 of Cause Code 85-3 outages did not ave a single ligtning strike located witin 5 NM and 48 ours before te failure. As a result, it is very unlikely tat convective weater is responsible for tose outages. Te Non-Correlated Strikes were obtained by processing 621 randomly selected outages during 24 monts (also randomly selected), wic led to a total number of 1,800,000 Non-Correlated Strikes. We ten created te 9 sets of clusters wit te different lengts (48 ours, 24 ours, 12 ours, 6 ours, 2 ours, 1 our, 30 minutes, 15 minutes and 5 minutes) and computed a binary logit model for eac of tem. Table 6 is an example of te models for clusters of 1 our. In tis version, equipment-types parameters are not included as tey proved to ave poor statistical significance. Ten, all te outages contained in te FAA data were filtered to consider only outage codes were ligtning strikes could be te original cause (Table 7) and Time and Space Windows were created for tem. Unsurprisingly, te correlation levels observed are lower tan for te Cause Code 85-3 outages, 27 % vs. 80%. Finally, we applied te 9 statistical models to te corresponding sets of clusters and used te aggregate metod presented above (Table 10). Te total number of outages reported under oter codes but considered by te model as caused by ligtning strikes spans between 1381 and 3013 wic implies a significant modification of te original number. Several reasons can explain te apparent differences between te original data and te output of te model. First, even wen te cause of te outage is not clear, people in carge of te maintenance are likely to indicate ligtning strikes if some convective activity was present in te region. Secondly, te FAA Cause Code system indicates te ultimate cause of te outage but does not indicate its early cause. For example, for outages reported under Power Supply (Cause Code 80-3), noting indicates wat actually caused te loss of power. Te first effect tends to overestimate te number of outages caused by ligtning strikes wereas te second one underestimates it. However, according to te results of te model, te magnitude of te second effect is larger (plus versus minus 200 for te first effect). Table 7: List of outage codes considered for te estimate Code Number Category Sub Category Equipment Power Supply Equipment Unable to Determine Cause 80-F 214 Equipment Facility Power and Support Systems Non-FAA Lines/Circuits Power Non-FAA Lines/Circuits Environmental Causes Non-FAA Lines/Circuits Unknown Prime Power Standby Power Interference Conditions Unknown - Radio frequency interference 6

7 Table 8: Summary of te number of outages considered for te different steps Model Creation Number of 85-3 outages (after cleaning) 987 Number of correlated outages 787 Results (Oter Codes) Total number of outages Number of outages after cleaning and code selection Correlated outages 7299 List of outages Outage 1 Outage Table 9: Exmaple of aggregate model min 15 min 5 min Total Outage n Table 10: Number of confirmed outages using te aggregate metod Number of models confirming te outage Cumulative number of confirmed outages All V. CONCLUSION Te correlation between ligtning strikes and outages exists and can be captured by statistical metods. To acieve tis goal, it is crucial to take into account te important data inaccuracies and to focus more on periods of ig convective activity rater tan on specific strikes to determine te causes of outages. Bot strike localization issues and imprecise outage report times imply a ig incertitude in te Target Strikes selection and forced us to introduce te Time and Space Window: a spatiotemporal area of 5 nautical miles and 48 ours associated wic eac outage and inside wic all strikes are considered as a potential cause of outages. Even wit tose large values, for 22% of te outages reported as caused by ligtning strike (Cause Code 85-3) we were not able to find a single ligtning strike inside te Time and Space Windows. In tese cases, it is very likely tat tese outages are not related to convective weater and teir FAA outage code sould probably be modified. Because of te previous definition of te Time and Space Window, numerous strikes ributed over large periods of time migt be responsible for a given outage and no caracteristics allow a reliable selection. For tis reason, we favored an approac based on clusters gatering ligtning strikes over periods of time spanning from 5 minutes to 48 ours to create independent regular binary logit models. Ten, we applied tose models to a selection of codes dealing wit power supply failure and unknown causes. Te resulting subset contained about 27,000 outages among wic 7,300 ad at least one ligtning strike inside teir Time and Space Window. In a number of cases spanning from 1381 to 3013 outages, ligtning strikes are likely to be te initial outage cause altoug tese numbers sould be carefully considered. Finally, two major sources of inaccuracy limit te precision of te models. Te first and easiest to fix is te localization precision of te strikes wic cause te correlation of too large of a number of ligtning strikes. More compreensive data providing te exact location of eac stroke is available and would reduce te Space window to an average of 500 meters. Te second issue is te outage detection time tat can be only partially solved via a list of all te systems featuring an automatic report device. In tese cases, te Time Window would srink to a few minutes and tus limit te number of Correlated Strikes. In addition, te list of systems using te same power source on some airports and te list of systems featuring forms of protection against ligtning strikes would also improve te overall precision. REFERENCES [1] E. G. Bates, T. A. Seliga, and T. Weyrauc, Implementation of grounding, bonding, sielding and power system improvements in FAA systems: reliability assessments of te terminal Doppler weater radar (TDWR), Digital Avionics Systems, DASC. Te 20t Conference, October 2001, vol. 1, pp. 2A5/1-2A5/7. [2] J. A. Cramer, K. L. Cummins, A. Morris, R. Smit, and T.R. Turner, Recent upgrades to te U. S. national ligtning detection network, 18t International Ligtning Detection Conference, Helsinki, Finland, 7-9 June, [3] K. L. Cummins, E. P. Krider, and M.D. Malone, Te U.S. national ligtning detection network and applications of cloud-to-ground ligtning data by electric power utilities, IEEE Trans. Electromagn. Compat., vol. 40, no 4, November