Improving Patent Examination Efficiency and Quality: An Operations Research Analysis of the USPTO, Using Queuing Theory.

Transcription

1 Imroving Patent Examination Eiciency and Quality: An Oerations Research Analysis o the USPTO, Using Queuing Theory By Ayal Sharon and Yian Liu Aendices APPENDIX I FOUNDATIONAL FORMULAS. Formula or Mean Length o Queue (L) Let π j be the steady-state robability that j items will be resent. I ρ<, then we have steady-state robability π j = ρ j ( ρ), so the mean length o the queue is λ L= jπ j =. μ λ j= 0 2. Priority Queue Formulas We assume n tyes o alications with distinct riority levels. We label the highest riority level as level, and lowest as level n. For level k, denote the arrival rate as λ k, and service rate as μ k. The traic intensity or the entire system is J.D. Candidate, George Mason University School o Law, M.B.A., B.S.E.E. The author is a Patent Examiner in the art o Simulation, Emulation, and Modeling (Class 703) at the U.S. Patent and Trademark Oice. This article does not reresent the views o the U.S. Patent and Trademark Oice. Assistant Proessor, George Mason University, Deartment o Systems Engineering & Oerations Research. See WAYNE L. WINSTON, OPERATIONS RESEARCH: APPLICATIONS AND ALGORITHMS 074 (4th ed. 2004) [hereinater WINSTON]. Electronic coy available at: htt://ssrn.com/abstract=026372

2 ρ = λ μ+ λ2 μ λn μn. 2 Remember that only i ρ< can the queue reach steadystate. Otherwise, it becomes saturated and overwhelms the system. When ρ<, the mean steady-state number o tye k items in the queue is L qk. The mean steady-state number o tye k items in the system is L k. The mean waiting time sent in the queue by a tye k item is W qk, and the mean waiting time sent in the system by a tye k item is L k. Then rom queuing theory: 3 W qk n 2 λ ( ) 2 kesk / k = = ( a )( a ) k k L = λ W qk k qk Wk = Wqk + μk L = λ W k k k where a 0 = 0 and: a k k i =. i= λ μ i These ormulae hold when all tyes o items have dierent riority. However, when a ew dierent tyes o items have the same riority, with dierent arrival and service rate, the situation will be more comlicated. The derivation o those ormulae ollows. Suose we have n dierent riority levels, and in each level k, we have m k dierent tyes o items with mean arrival rate λ kj and mean service rate μ kj, j=,,m k. 2 See id. at 27 (reerencing the equation above eq.62). 3 Id. at 27, Electronic coy available at: htt://ssrn.com/abstract=026372

3 Then, we ind the total arrival rate, weighted mean service rate, and square o service time or each riority level, which are m k λk = λkj j= μ k = = mk μ kjρ j= kj mk ρ j kj ES 2 ( k ) mk 2 ES ( ) kj ρ j= kj mk ρ j kj = = where ρkl = λkj μkj is the weight or tye j item in level k. Then, using the ormulae or distinct riority levels and noticing that ES ( ) = 2 μ, we get the ormulae needed or our roblem. 2 2 kj kj W qkj n 2 λ ( ) 2 kesk / k = = ( a )( a ) k k L = λ W qkj kj qkj W kj = Wqkj + μ kj L = λ W. kj kj kj where a k, λ k, μ k and ES are as deined above. 2 ( k ) 3

4 APPENDIX II PARAMETERS USED IN THE MODEL In this section, we resent all the arameters that we use in the ollowing sections o the model. Most o the arameters come directly rom ublished data, while a ew are based on reasonable assumtions. Alication Descrition Derivation Value Data Reg New Regular New Alications See ootnote 4 254,000 RCE Total RCEs See ootnote 5 53,000 st RCE st RCE See ootnote 6 43,000 2nd RCE 2 nd or Subsequent RCEs See ootnote 7 0,000 Cont Total Continuations + Divisionals See ootnote 8 63,000 st Cont st Continuation See ootnote 9 32,700 2nd Cont 2 nd or Subsequent Continuations See ootnote 0,800 Divs Divisional Alications See ootnote 8,500 Secial New Secial New Alications See ootnote 2,200 Aeal Aealed Alications See ootnote 3 2,834 4 See Changes to Practice or Continuing Alications, Requests or Continued Examination Practice, and Alications Containing Patentably Indistinct Claims, 7 Fed. Reg. 48, 50 (roosed Jan. 3, 2006) (to be codiied at 37 C.F.R. t. ) (roviding data that in iscal year 2005, the USPTO received 37,000 nonrovisional alications, o which aroximately 63,000 were continuing alications). 5 Id. 6 See id. (roviding data that USPTO received 53,000 total requests or continued examination, o which, 0,000 were second or subsequent requests). 7 Id. 8 Id. 9 See id. (roviding data that USPTO received 63,000 total continuing alications, o which 8,500 were divisional alications and,800 were second or subsequent continuation/cip alications). 0 Id. Id. 2 Changes to Practice or the Examination o Claims in Patent Alications, 7 Fed. Reg. 6, 63 (roosed Jan. 3, 2006) (to be codiied at 37 C.F.R. t. ). 3 U.S. PATENT AND TRADEMARK OFFICE, PERFORMANCE AND ACCOUNTABILITY REPORT FISCAL YEAR 2005, TABLE 4: SUMMARY OF CONTESTED PATENT CASES (2006), available at htt:// 4

5 Other Descrition Derivation Value Published Parameters N Number o examiners See ootnote 4 4,25 Β Probability Examiner airmed at re-aeal See ootnote c Mean number o non-inal rejections er arent alication See ootnote 6.6 Personally Descrition Derivation Value Estimated Parameters P Probability o only one invention in an Personal estimation 0.98 alication m Probability alicant iles ater-inal Personal estimation 0.20 amendment (i 2 / i ) Ratio o robability a atent issues in the Personal estimation.5 second or later round o rosecution, divided by robability it issues in the irst round x Percentage o alications abandoned ater the inal rejection (rather than ater a non-inal rejection) Personal estimation 0.95 Calculated Descrition Derivation Value Parameters hr Examiner hours er year = 48 weeks *, hours/week λ Hourly arrival rate o new alications er = Reg New / examiner (N*hr) d Mean number o inventions er alication = + (Divs / (Reg New st Cont + 2nd Cont + RCE)) s Probability alication iled with etition to = Secial New / Reg make secial New C Exected number o st generation = st Cont / (Reg New continuation children created during non-inal + RCE + Aeal/ β) status M 2 Mean number o 2nd or subsequent generation o continuation children er new alication = 2nd Cont / Reg New See John Doll, Comm r or Patents, U.S. Patent and Trademark Oice, Address at Chicago Town Hall Meeting 20 (Feb., 2006), available at htt:// (ower oint resentation) (N = FY 05 BOY Examiner Sta + FY 05 Hiring - FY 05 Attrits). 5 See generally U.S. PATENT AND TRADEMARK OFFICE, PATENT PUBLIC ADVISORY COMMITTEE MEETING 4 45 (Oct. 25, 2005), available at htt:// 6 See OFFICE OF INSPECTOR GEN., U.S. DEP T. OF COMMERCE, FINAL INSPECTION REPORT NO. IPE- 5722/SEPTEMBER 2004, USPTO SHOULD REASSESS HOW EXAMINER GOALS, PERFORMANCE APPRAISAL PLANS, AND THE AWARD SYSTEM STIMULATE AND REWARD EXAMINER PRODUCTION, 7 ig.2 (2004), available at htt:// (actions er alication extraolated to 2005). 5

6 We have ive other arameters to be estimated (),, () e, e, and a. Because these arameters cannot be derived directly rom the data, we set u ive equations containing these ive unknown variables rom the data and solve or them. First, we have 43,000 irst round RCEs or 254,000 new alications and 32,700 +,800 child continuation alications, which means: () () () E = d e = 43000, lug in d =.0526, we get: = () () e (Eq. ) Second, we have 0,000 second and later rounds o RCE or these 298,500 alications, which means: d + (2 ) () () e E = ( d e ) 0000, = d e lug in the value o d and (Eq. ), we have: d e = , d e d = 0.886, e hence: = e (Eq. 2) Third, the robability that a atent issues in the second or later round o rosecution (denoted by i 2 ) is higher than the robability it issues in the irst round 6

7 (denoted by i ). We ersonally estimated this ratio as.5. That is, i 2 =.5i. Also, since the average number o oice actions or each round is estimated to be 2.6: 7 ( i ) = 2.6 () ( i ) = That is to say: i () = log 2.6, i 2 = log 2.6. Since i 2 =.5i, we have: log =.5( log ), () which yields: ( ) = 2.6 () (Eq. 3) Fourth, we have 2834 aeals sent to BPAI, which means: () () () Aβ = d + d e 2834 aβ =, d e lug in β, d and equations (),, we get: () ( ) a = (Eq. 4) 7 See id. (actions er alication extraolated to 2005). This value is conirmed by anecdotal evidence. See Iowa State University Research Foundation, Inc. Oice o Intellectual Proerty & Technology Transer, Patents and the Patent Process 7 (Feb. 3, 2007) ("How many oice actions will occur? It is reasonable to exect to receive three oice actions er alication. I continuations are iled, or examle, there will be additional oice actions issued.") htt:// ocess.cm. 7

8 Fith, we have 96,76*.=05,794 8 abandoned alications. We estimate that the ercentage o alications abandoned ater the inal rejection (rather than ater a noninal rejection) is x = 95%, which means: () () ( a ) e ( a e ) ( ) 05794* , + a = = e β lug in equations () and, and ater some arrangement, we get: () ( )( a) = (Eq. 5) Solving equations (Eq.) to (Eq.5), we get the ive arameters listed in the ollowing table: Solved Parameters Descrition Value () ρ robability to reach inal rejection in the irst round ρ robability to reach inal rejection in the second or later round () ρ e robability to choose RCE ater the irst round ρ e robability to choose RCE ater the second or round robability to choose aeal a 8 U.S. PATENT AND TRADEMARK OFFICE, PERFORMANCE AND ACCOUNTABILITY REPORT FISCAL YEAR 2005, TABLE : SUMMARY OF PATENT EXAMINING ACTIVITIES (2006), available at htt:// (abandonment data rom 2003 extraolated to 2005). 8

9 APPENDIX III THE DETAILED MODEL. Estimating the Number o Arrivals, Sorted by Tye o Alication Our roblem is a riority queuing model with branching eedback. Since there are several dierent riority levels and each level receives dierent tyes o alications, it is necessary to estimate searately the arrival rate o each tye o alication. In order to calculate the arrival rates o the dierent tyes o alications, we make the ollowing assumtions: () we deine a round o rosecution as all the noninal rejections until, and including, the inal rejection; there can be several non-inal rejections beore a inal rejection is issued; (3) ater a inal oice action, the alicant may choose to RCE, aeal, or abandon; and (4) once the alicant chooses to aeal, there are tyically no more RCEs (desite the rule that alicants can ile an RCE i the BPAI airms the Examiner s rejection). A. RCE For each brand new alication, the exected number o RCEs it will generate ater the irst round o rosecution is: E = d () () () e where d is the exected number o inventions in an alication and () is the robability or an alication to reach the ost-inal stage in the irst round (i.e., the robability the alication will not be allowed or abandoned beore a inal-decision). The arameter round o rosecution. () e is the robability that an alicant will ile an RCE ater the irst 9

10 For the sake o the simlicity o the model, we assume that these robabilities remain constant or subsequent rounds o rosecution 9 and denote them as and e resectively or all rounds starting ater the second round. The exected total number o RCEs ater the second round o rosecution is: E = d d () () e e and similarly, or all subsequent rounds, is: E = d ( d ). ( k) () () ( k ) e e We take the sum o the series and we have the number o RCEs that the USPTO roosed reorm is targeting to eliminate: d (2 ) () () E + = d e. d e e Thereore, the total number o RCEs currently generated by a new alication is: () (2 + ) () () E = E + E = d e. d e B. Aeal An alicant can ile an aeal immediately ater the irst round o rosecution, or alternatively, can aeal ater one or more additional rounds o rosecution. (The alicant obtains additional rounds by iling RCEs). 20 For simlicity, we neglect the instances when an alicant iles an aeal more than once in the course o rosecuting 9 There is an argument that this is not the case. It may be that in subsequent rounds o rosecution, the robability o allowance rises, so the robability o inal rejection alls. Also, it is ossible that the robability o the alicant iling an RCE alls in the second and subsequent rounds. 20 Changes to Practice or Continuing Alications, Requests or Continued Examination Practice, and Alications Containing Patentably Indistinct Claims, 7 Fed. Reg. 48, 50 (roosed Jan. 3, 2006) (to be codiied at 37 C.F.R. t. ). 0

11 an alication. 2 We assume that an alicant iles an aeal only once. Suose at the stage o inal-rejection, the robability to choose aeal is a, then, or each brand new alication, the exected number o aeals it will generate (ignoring any ossible child continuation alications) is A= d + E. () a a C. Continuations Parent Alication Remaining Alive Regular new alications, RCEs, and aealed alications can all sawn child continuation alications. Some o the arent alications remain alive in rosecution and eed back into the arrival queue i the continuation is iled immediately ater receiving a non-inal rejection. In the case o aealed alications, this can haen i a re-aeal conerence overturns the examiner. 22 I we let n be the number o arent alications that are alive in each round due to non-inal rejections, then the total number o living arent alications is D. Continuations Child Alications P= ( d + E+ A) n Now we consider child alications. For each tye o alication (regular new, RCE, or aeal), suose the exected number o irst generation children it generates is c. Then or each brand new alication, the exected total number o irst generation children is: 2 Multile aeal bries can be iled in an alication when the examiner reoens rosecution ater being reversed at a re-aeal conerence, ollowed by the alicant iling another aeal. 22 About 60% o re-aeal conerences result in the reversal o the examiner. See generally U.S. PATENT AND TRADEMARK OFFICE, PATENT PUBLIC ADVISORY COMMITTEE MEETING 4 45 (October 25, 2005), available at htt://

12 M = ( d + E+ A) c. For simlicity, i we let M 2 denote the total number o second and lower generation o children, then the total number o osring including itsel is + M+ M Priority Queue Arrival and Service Rates Most queuing models have two very imortant arameters: mean arrival rate (λ) and mean service rate (μ). 23 Mean arrival rate is the average rate at which items arrive, and is measured in items er hour or some similar metric. 24 Mean service rate is the rate at which the server comletes rocessing arriving items. 25 The inverse o the mean service rate is the mean service time (/μ). Both the arrival rate and service rate are random variables. In order to simliy our roblem, we assume no bulk arrivals (i.e., multile items arrive at exactly the same instant), and no memory (ast arrivals do not aect uture arrivals). According to robability theory, in this situation, the number o arrivals in a unit time length its a Poisson distribution with mean λ. 26 Thereore, the number o arrivals in any time interval o length t is λt. 27 Regarding the service rate, or comutational simlicity, we assume the no-memory roerty, so that the service time ollows an exonential distribution with 23 EDWARD D. LAZOWSKA ET AL., QUANTITATIVE SYSTEM PERFORMANCE 5 (Prentice-Hall 984) (reers to mean arrival rate as workload intensity and to mean service rate as throughut ), available at htt:// By convention, the symbol or mean arrival rate is (λ), and or mean service rate is (μ). 24 Id. 25 Id. 26 WAYNE L. WINSTON, OPERATIONS RESEARCH: APPLICATIONS AND ALGORITHMS, (4th ed. 2004). 27 Id. 2

13 arameter μ. That is, the mean service time or one item is μ - and mean service rate (number o items served in unit time) is μ. 28 For Poisson distribution, the answer is already given in the aer written no bulk arrivals (i.e., multile items arrive at exactly the same instant) and no memory (ast arrivals do not aect uture arrivals these are the reasons or Poisson distribution, and these assumtion are roughly true in reality. These two actors, λ and μ, directly inluence the queue length (L) and the waiting time (W). The queue length (L) grows over time i λ μ. In other words, the queue grows when the mean arrival rate is larger than the mean service rate. This reresented by the traic intensity variable ρ = λ / μ. When ρ (in other words, λ μ), then no steady-state can be reached and the system is overwhelmed. I, on the other hand, ρ <, then we can calculate the mean queue length (L). 29 In addition, Little s Law, L = λw, gives the relation between mean queue length and mean waiting time. 30 Based on the results in the revious section, we can list the arrival rate and service rate or all item tyes in the riority queue as shown in the ollowing sections. A. Priority Level Secial Amended Docket We assume a one-hour service time or all items in this riority level, thus μ - = or all the dierent tyes o alications in this riority level. For ater-inal amendments, the mean arrival rate and mean service time are as ollows: 28 See id. at The ormula or mean length o queue (L) can be ound on age three o the aendix. 30 See WINSTON, sura note 26, at See also LAZOWSKA, sura note 23, at

14 λ = λ , () ( M M2)( d ( E A) ) m μ =, where m is the robability alicants ile ater-inal amendments. For BPAI Decisions: λ = λ β, () 2 ( M M2)( d ( E A) ) a μ 2 =, where β is the robability an aeal is sent to the BPAI. B. Priority Level 2 Regular Amended Docket We assume a ten-hour service time or all items in this riority level, thus μ - = 0 or all the dierent tyes o alications in this riority level. For resonses to non-inal rejections: λ = λp( + M + M ), 2 2 For RCEs: μ 2 = 0. λ = λe( + M + M ), 22 2 For aeal bries: μ 22 = 0. λ = λ β, () 23 ( M M2)( d ( E A) ) a μ 23 = 0. 4

15 For elections: λ = λ( + M + M )( + E)( ), 24 2 μ 24 = 0, where is the robability or an alication to have exactly one invention. For aeals: λ = λ A( + M + M ), 25 2 μ 25 = 0. C. Priority Level 3 Secial New Docket We assume a ten-hour service time or all items in this riority level, thus μ - = 0 or all the dierent tyes o alications in this riority level. The secial new docket contains three tyes o alications: () new alications that the alicants have etitioned to make secial; divisional alications; (3) and child alications. For cases etitioned to make secial, s is the robability or a new alication to be with etition to make secial. λ3 = λ s, μ 3 = 0, For divisional alications: λ = λ( + M + M )( + E)( d ), 32 2 μ 32 = 0. 5

16 For child alications (continuations and continuations-in-art): λ = λ( M + M ), 33 2 μ 33 = 0. D. Priority Level 4 Regular New Docket For regular new alications without etition to make secial: λ4 = λ( s ), μ 4 = 0. 6

17 APPENDIX IV SENSITIVITY ANALYSIS In this section, we discuss the sensitivity analysis results. The urose o sensitivity analysis is to determine which variables are the most imortant. In other words, the urose o sensitivity analysis is to determine to which arameters the model is most sensitive. We erormed sensitivity analysis on a small subset o the variables in our model. These included three ersonally estimated arameters: the robability that alications abandoned during rosecution are abandoned ater the inal rejection rather than ater a non-inal rejection ( x ), and the ratio o robability a atent issues in the second or later round o rosecution, divided by robability it issues in the irst round (i 2 / i ), and the robability alicant iles ater-inal amendment ( m ). In addition, and erhas most imortantly, we erormed sensitivity analysis on the mean number o non-inal rejections er arent alication (c ). Changing the value o m would not change the values o the ive solved arameters: (),, () e, e, and a, but would change the inal results. Changing any o the other three arameters ( x, c, or [i 2 / i ]) would change the values o both the ive solved arameters and the inal results. For each analysis, we could only do the sensitivity analysis or one arameter such as m while keeing the other arameters at their original values. Thereore, in the tables below, we modiy the values or only one arameter or each scenario. From the tables below we can see that or each ixed arameter, our conclusion still holds: restricting rounds o continuing alications (children) or RCE will not hel to reduce the saturation o the atent system. The sum o the rhos in the riority queues 7

18 remains greater than one, which means the system is saturated. The greatest contributor to the traic intensity is consistently ρ 2, which is the regular amended docket. Reducing the arrival rate o items in this riority level is key to reducing the traic intensity o the entire system. Reducing the number o non-inal rejections in each round o rosecution hels substantially and brings the sum o the rhos below the critical value o one. Changing the other arameters ( x, m, or [i 2 / i ]), such as limiting the number o RCEs to one, or reducing the non-inal rejection, etc., has very little eect on the end result. These are non-sensitive arameters. The arameter c, on the other hand, has a great eect on the inal result. It is a sensitive arameter. This matches our original result, that the number o non-inal rejection er round o rosecution is a key issue. It clearly makes a dierence when the number varies between.2 and 2.0 (original was.6 or non-inal, and 2.6 or non-inal lus inal, i.e..6+ = 2.6). 8

19 . Sensitivity analysis o m Changing m doesn t aect the arameters (),, () e, e, and a. Pre-reorm status quo: m =0. m =0.5 m =0.2 m =0.25 m =0.3 Ρ Ρ Ρ Ρ 4 Sum o rho Allowing one round o RCE: m =0. m =0.5 m =0.2 m =0.25 m =0.3 ρ Sum o rho Allowing no RCE: m =0. m =0.5 m =0.2 m =0.25 m =0.3 ρ Sum o rho Allowing one generation o continuing alications (children): m =0. m =0.5 m =0.2 m =0.25 m =0.3 ρ Sum o rho Allowing no continuing alications (children): m =0. m =0.5 m =0.2 m =0.25 m =0.3 ρ Sum o rho

20 Reducing the number o non-inal rejections to one er round o rosecution: m =0. m =0.5 m =0.2 m =0.25 m =0.3 ρ Sum o rho

21 2. Sensitivity analysis o (i 2 / i ) Changes to the ive solved arameters: (i 2 / i ) =.3 (i 2 / i ) =.4 (i 2 / i ) =.5 (i 2 / i ) =.6 (i 2 / i ) =.7 () ρ ρ () ρ e ρ e * a Pre-reorm status quo: (i 2 / i ) =.3 (i 2 / i ) =.4 (i 2 / i ) =.5 (i 2 / i ) =.6 (i 2 / i ) =.7 ρ Sum o rho Allowing one round o RCE: (i 2 / i ) =.3 (i 2 / i ) =.4 (i 2 / i ) =.5 (i 2 / i ) =.6 (i 2 / i ) =.7 ρ Sum o rho Allowing no RCE: (i 2 / i ) =.3 (i 2 / i ) =.4 (i 2 / i ) =.5 (i 2 / i ) =.6 (i 2 / i ) =.7 ρ Sum o rho Allowing one generation o continuing alications (children): (i 2 / i ) =.3 (i 2 / i ) =.4 (i 2 / i ) =.5 (i 2 / i ) =.6 (i 2 / i ) =.7 ρ Sum o rho

22 Allowing no continuing alications (children): (i 2 / i ) =.3 (i 2 / i ) =.4 (i 2 / i ) =.5 (i 2 / i ) =.6 (i 2 / i ) =.7 ρ Sum o rho Reducing the number o non-inal rejections to one er round o rosecution: (i 2 / i ) =.3 (i 2 / i ) =.4 (i 2 / i ) =.5 (i 2 / i ) =.6 (i 2 / i ) =.7 ρ Sum o rho

23 3. Sensitivity analysis o c Changes to the ive solved arameters: c =.2 c =.4 c =.6 c =.8 c =2.0 () ρ ρ () ρ e ρ e a Pre-reorm status quo: c =.2 c =.4 c =.6 c =.8 c =2.0 ρ Sum o rho Allowing one round o RCE: c =.2 c =.4 c =.6 c =.8 c =2.0 ρ Sum o rho Allowing no RCE: c =.2 c =.4 c =.6 c =.8 c =2.0 Ρ Ρ Ρ 4 Sum o rho Allowing one generation o continuing alications (children): c =.2 c =.4 c =.6 c =.8 c =2.0 Ρ Ρ Ρ Ρ 4 Sum o rho

24 Allowing no continuing alications (children): c =.2 c =.4 c =.6 c =.8 c =2.0 Ρ Ρ Ρ Ρ 4 Sum o rho Reducing the number o non-inal rejections to one er round o rosecution: c =.2 c =.4 c =.6 c =.8 c =2.0 Ρ Ρ Ρ Ρ 4 Sum o rho

25 4. Sensitivity analysis o x Changes to the ive solved arameters: x = 90% x = 92.5% x = 95% x = 97.5% x = 99.5% () ρ ρ () ρ e ρ e a Pre-reorm status quo: x = 90% x = 92.5% x = 95% x = 97.5% x = 99.5% ρ Sum o rho Allowing one round o RCE: x = 90% x = 92.5% x = 95% x = 97.5% x = 99.5% ρ Sum o rho Allowing no RCE: x = 90% x = 92.5% x = 95% x = 97.5% x = 99.5% ρ Sum o rho Allowing one generation o continuing alications (children): x = 90% x = 92.5% x = 95% x = 97.5% x = 99.5% ρ Sum o rho

26 Allowing no continuing alications (children): x = 90% x = 92.5% x = 95% x = 97.5% x = 99.5% ρ Sum o rho Reducing the number o non-inal rejections to one er round o rosecution: x = 90% x = 92.5% x = 95% x = 97.5% x = 99.5% ρ Sum o rho