Application Bayesian Approach in Tasks Decision-Making Support

Size: px

Start display at page:

Download "Application Bayesian Approach in Tasks Decision-Making Support"

Rafe Atkins
5 years ago
Views:

1 America Joural of Applied Mathematics ad Statistics, 05, Vol 3, No 6, 57-6 Available olie at Sciece ad Educatio Publishig DOI:069/ajams Applicatio Bayesia Approach i Tasks Decisio-Makig Support Aliyeva Taraa Abulfaz Departmet of Iformatio Ecoomic ad Techology, Azerbaija State Ecoomic Uiversity, Baku, Azerbaija Correspodig author: alilitaraa@gmailcom Abstract The proposed work is devoted to the use of the Bayesia approach based o probabilistic method of use, alog with the origial statistical data prior iformatio about the curret process Especially sigificat advatages over classical methods i terms of the accuracy of the statistical iferece Bayesia approach is i the relatively small samples, which is very characteristic of the simulatio A theorem o miimizig classificatio error probability is proved About choosig a priori distributio parameters from the coordiated class of distributios is cosidered Keywords: a priori distributio, a posteriori distributio, a coordiated distributio, Bayesia approach Cite This Article: Aliyeva Taraa Abulfaz, Applicatio Bayesia Approach i Tasks Decisio-Makig Support America Joural of Applied Mathematics ad Statistics, vol 3, o 6 (05): 57-6 doi: 069/ajams Itroductio May statistical problems idepedetly of the methods of solvig them have a commo property: before you get a specific set of data as a potetially acceptable to study the situatio examies the several probabilistic models Whe the data is received, there is a proouced i a form of kowledge about relative acceptability of these models Oe way to "revisio" o the relative acceptability of probabilistic models is the Bayesia approach, which is based o Bayes' theorem The Bayesia approach has a umber of advatages that make it attractive eough for widespread applicatio The mai differece of this approach from other approaches is that before the data is received, the decisio-maker or a statisticia examies the degree of his cofidece i the possible models ad presets them i the form of probabilities Oce the data is received, the Bayes' theorem allows us to calculate a ew set of probabilities, which are revised the degree of credibility of possible models takig ito accout the ew iformatio received through the data As is kow, the statistical datas are ofte abset i actual tasks Decisio-Makig support, which makes use of may traditioal frequecy approaches ulawful Available iformatio may oly cotai subjective assessmets i the form of expert evaluatios ad judgmets Moreover, the situatio i which it is decided, may be geerally ew at all ad has ever bee previously aalyzed These characteristics complicate the tasks Decisio-Makig support process ad may call ito questio ay coclusios ad opiios I such a situatio, the applicatio of the Bayesia Approach [] is very effective Experimetal Methods The Bayesia approach [] is based o the statistical ature of the observatios It is kow that i the decisio statistic used as iput simultaeously two types of iformatio: a priori (before the existig statistical productio) ad is cotaied i the iitial statistical data I this case, a priori iformatio give to him i the form of a a priori probability distributio of the aalyzed ukow parameter, that describes the degree of cofidece that this optio will take some value, before the collectio of baselie statistics State the value of the ukow parameter of the system meas to fid its posteriori distributio The idea of the Bayesia approach lies precisely i the trasitio from a priori distributio to the posterior distributio Let A, A,, A - full group of mutually exclusive evets i = A i = Ω, Ai Aj =, whe i j The the posterior probability is: PA ( i) PBA ( i) PAB ( i ), PA ( ) ( ) i i PBA = i = () where PA ( i ) - the a priori probability of the evet Ai, PBA ( i) - the coditioal probability of a evet B, provided that the evet occurred A i, ad the evet B has a o-zero probability ( PB ( ) > 0) Iitially, Bayesia classificatio is used to formalize the kowledge of experts i expert systems, Bayesia classificatio is ow also used as a method of Data Miig The Bayesia approach cosists of the followig steps:

2 58 America Joural of Applied Mathematics ad Statistics Determiatio of the a priori distributio p ( ) of the desired multi-dimesioal parameter ; Gettig the source of statistical data x, x,, x to the laws of distributio f( x ), i =,,, at fixed parameter ; 3 The calculatio of the likelihood fuctio Lx (, x,, x ) defied by the relatio L( x, x,, x ) f( x ) f( x ) f( x ) i = () 4 The calculatio of the posteriori distributio ( x, x,, x ), determied by the relatio (3) p( ) Lx (, x,, x ) p( x, x,, x ) = Lx (, x,, x ) p( ) d 5 Buildig a Bayesia poit ad iterval estimates of parameters ˆ usig the mea or modal value of the distributio foud from the formula (3) ˆ mea E( x, x,, x ) p( x, x,, x ) d = = (4) ˆ mod = argmax ( x, x,, x ) Experimetal Set Up I may decisio-makig tasks a priori probability iformatio about the States of ature ca be chaged after the ew expert assessmets or as a result of observatio relevat developmets related to states ad cofirmig or refutig a priori iformatio The depedece of the posterior probabilities of a priori shows how much iformatio ad values of the ukow parameters cotaied i the statistics If the posterior probabilities are highly depedet o the a priori, it is likely the data cotais little iformatio If the posterior probabilities are weakly depedet o the choice of a priori distributio, the data are iformative Thus, usig the Bayesia approach except for the probability distributio of the cosidered radom variable X is assumed that some a priori distributio parameters, the distributio fuctio of values X Relyig o statistical data, the a priori distributio of the parameters is modified by multiplyig the likelihood fuctio ad ormalizatio The result of the modificatio is the posterior distributio of the parameters I other words, the parameters of the radom variable themselves are radom variables with some distributio The most importat ad challegig at the same time is a matter of choosig a priori distributio parameters Oe of the factors here is that the presece of eve statistical iformative "poor" a priori distributio will ot sigificatly affect the posterior Aother importat factor is the computatioal complexity, especially if the calculatios of the posterior distributio produced i series as they become available statistical iformatio Therefore, the choice of the a priori distributio affects its belogig to so-called class coordiated distributios such distributios that are a priori ad a posteriori the same distributio, but with differet parameters Priori distributios modelig o a priori iformatio are called uiformative The postulate Bayes-Laplace [3] says that whe othig is kow about the pre-parameter a priori distributio should be uiform, ie, all maer of outcomes of a radom variable have equal probability The mai problem of the use of uiform distributio as a o-iformative a priori distributio is that a uiform distributio is ot ivariat with respect to the fuctio of the parameter If we kow othig about the parameter, the we also do ot kow, for example, about the fuctio / However, if has a uiform distributio, / o loger has a uiform distributio, although accordig to the postulate Bayes - Laplace, / must have the uiform distributio Moreover, the uiform distributio caot be used as a priori, if the set parameter values are ifiite I the literature, there is eough cosiderable umber of approaches for the selectio of the o iformatio priori distributio havig its advatages ad disadvatages However, the most iterestig is, the approach is completely differet from most traditioal The essece of this approach is as follows Defie more tha oe prior distributio, but a whole class Μof distributios π, for which you ca fid lower ad upper probability of the evet A as a { π } PA ( ) = if Pπ ( A): Μ{ π } PA ( ) = sup Pπ ( A): ΜUder certai coditios, the set is completely determied by the lower ad upper probability distributio fuctios It should be oted that the class should be see "ot as a class of suitable prior distributios, as well as a suitable class of prior distributios This meas that every sigle distributio of the class is ot the best or "good" a priori distributio, sice o sigle distributio ca ot satisfactorily simulate the absece of iformatio But the whole class as a whole, defie the upper ad the lower the probability distributio is a appropriate model for the lack of iformatio Defiitio The family of a priori distributios G = p( ; D) is called the cojugate with respect to the { } likelihood fuctio ( ) L x x x, ad if the,,, p( x, x,, x ) posterior distributio as calculated by the formula (3), agai belogs to the same family G If the likelihood fuctio L( x, x,, x ) ca be represeted i a form (,,, ) L x x x T( x, x,, x ),, = v ψ ( x, x,, x ), Tm( x, x,, x); Where T( x, x,, x ), j =, m ad ψ ( x, x,, x ) - some of the fuctios of the observatios x, x,, x, do ot deped o the parameters, the there is the family G = p( ; D) a priori distributios cojugate to { } ( ) L x, x,, x The fuctios T( x, x,, x ), j =, m also called sufficiet statistics O the other had, before ay data or observatios of parameters are ofte ot kow, ie the researcher does (5)

3 America Joural of Applied Mathematics ad Statistics 59 ot have ay useful a priori iformatio about the values of the estimated parameter I such cases it is ecessary to cosider the followig selectio rules [4] priori distributio: If the estimated scalar parameter ca take values i a fiite iterval [ mi ; max ] or a ifiite iterval from to, the the a priori desity fuctio p ( ) should be cosidered costat at the appropriate iterval; if the meaig of the estimated parameter follows that it ca take ay positive values, it should be cosidered costat o the real lie desity fuctio distributio of the logarithm of the value of the parameter, i e, p (l ) = cost, whe (0; ) This a priori distributio is deoted by ( ) The fact that so defied o ifiite direct or semi-direct a priori distributios violated kow rules of ormalizatio of the probability desity fuctio (because the ( ) d, ( ) d =, where the itegratio is over all possible values), does ot deliver "techical icoveieces" As a priori distributios used uiform distributio (l ) = cost ad it allows the desity distributio fuctio ( ) to miimize the etropy measure of a iformatio - H = p( )l p( ) d for arbitrarily large a values a Give the ormalizatio of the desity fuctio (3) we obtai: (6) p( x, x,, x) ~ p( ) L( x, x,, x ) Us determie the form a priori desity p ( ) for the case p (l ) = cost, whe (0; ) Let F ( y) = P{ < y} - distributio fuctio of the parameter The { } { } l F ( y) = P < y = P l < l y = F (l y) Accordigly, the desity fuctio of the parameter is: F( y) Fl(l y) (l y) f ( y) = = y (l y) y = fl (l y) ~, y y sice by the coditio f l (l y) = p(l ) = cost So that for (0; ) we have: ( )~ (7) From this we obtai formula for the calculatio of the posterior distributio ( x, x,, x) ~ ( ) L( x, x,, x ), (8) where the likelihood fuctio Lx (, x,, x ) has the form (5) Remark Use as a priori laws of probability distributios associated with the observed geeral set of a family of sets p( ; D) However, the of priori distributios { } implemetatio of Bayesia approach is ecessary to operate the specific priori distributio, which requires kowledge of the umerical values D 0 of the parameters D o which our a priori probability distributio depeds I a broad sese, the a priori distributio parameters ca be determied by the method of momets with kow average values of the estimated parameters = E = ( E, E,, E ) T ad their stadard 0 s =, D p( ; D) deviatios D Let { } =,, s = D s, the family of priori distributios, cojugated with the likelihood fuctio Lx (, x,, x ) of observatios available to us ad let D 0 set parameter values D for the aalyzed case The, usig a series of idetical trasformatios the right side of the relatio (9) p( x, x,, x) ~ p( ; D0) L( x, x,, x ) is give to factors idepedet of to the species p( ; Dx (,, x )), where the last fuctio belogs to the family p( ; D), ad each of the compoets d ( x,, x )( j =,,, q) of the vector of parameters j (,, ) Dx x is a fuctio of 0 x,, x Cosistecy of distributios is determied ot oly by the views of a priori distributio, but also the view of the likelihood fuctio, ie, type of distributio must be maitaied whe multiplyig a priori distributio o the likelihood fuctio with the ormalizatio Dirichlet distributio also applies to distributios For a descriptio of the Dirichlet distributio, cosider the stadard polyomial model Let { ω ω } D ad { } Ω=,, m the set of possible outcomes, ad there is a set of N observatios idepedetly selected from Ω the same probability of each outcome { j} P ω = j for all j =,, m, where j 0 ad m j = The probability that the outcome ω j of the j= N observatios will occur N j time, determied from the well-kow formula of polyomial distributio with parameters,, m However, the parameters,, m themselves may be radom variables ad have a distributio or probability desity p( ) Oe of the most iterestig distributios of the parameters,, m is a Dirichlet distributio, which is coordiated with a polyomial distributio i the sese that a priori ad a posteriori distributios are Dirichlet distributios A priori Dirichlet distributio (,) st for a radom vector of probabilities = (,, m ), where t = ( t,, t m ), has the probability desity fuctio of the probability desity of the proposed de Groot (De Groot) [5] m m st ( ) () ( ) j p = Г s Г st j j j= j= Here the parameter is the average value (expectatio) the probability i ; the parameter s > 0 determies the

4 60 America Joural of Applied Mathematics ad Statistics effect of a priori distributio o the posterior probabilities; the vector t belogs to the ier area uit simplex of dimesio m, which we deote S(, m ) ; Г() - the gamma fuctio, satisfyig coditios Г( x+ ) = xг( x) ad Г () = Necessary to ote that the variables of the Dirichlet distributio are probabilities,, m, satisfyig the coditio S(, m) This meas that ot oly the radom evets are cosidered, but their probabilities After receivig the observatio vector = (,, m ), where j - the umber of observatios outcome ω j, multiplyig a priori desity o the likelihood fuctio L (, ), we obtai the posterior desity: ( ) m j+ st j p( ) L ( ) = j j= which ca be cosidered as the desity of the Dirichlet distributio ( N+ st, ), where j + st j t = ( t,, tm), tj = N + s I other words, the Dirichlet distributio belogs to the coordiated class of distributios ad uder the recalculatio a priori parameters t are coverted ito t This is a very importat feature, through which the Dirichlet distributio is widespread i the Bayesia aalysis 3 Results ad Discussio 3 Results Example a) A radom variable with ukow parameter value has a expoetial distributio Check the coditios of existece of the cojugate priori distributio, to determie its accessories thereof Class of distributios ad the specific values of the parameters of the distributio It is kow that x e for x 0, f( x ) = 0 for x < 0 From equatio () have: x i L ( x, x,, x ) ( ) i = f xi = e = Therefore, (0) x i L( x, x,, x ) = e for xi 0, 0 for xi < 0 Here m = ; T ( x, x,, x ) = x - sufficiet statistic i that cofirms the existece of a priori, cojugated with L( x, x,, x ) the distributio of the parameter Now will update the belogig of this distributio to a certai class of distributios Takig ito accout (7), from (8) we obtai: ( x, x,, x) ~ ( ) L( x, x,, x ) x i ~ e () The right side of relatio () states that the family of cojugate priori distributios parameter expoetially distributed the geeral total belogs to the class of gamma distributio with the desity α β α β p( ) = е, > 0 Г( α) ad distributio of parameters α = ad xi β =, respectively It is kow that the mea value ( E ) ad variace ( D ) of gamma - distributios are expressed by the parameters of the distributio α ad β by the formulas: α α E =, D = β β Substitutig ito these relatios istead E ad D respectively give values 0 ad, we obtai as the solutios of a system from the two equatios (ad relative to α ad β ): α 0 0 =, β = I the example above, we defie a posteriori distributio, takig ito accout (0) ad the desity of gamma distributio: x i α β ( x, x,, x ) ~ e e From here, it is see that the a posteriori distributio of the parameter agai obeys the law gamma - distributio, but with the followig parameters: = +, = + xi α α β β b) The radom variable (the umber of successes i s series of Beroulli trials) ξ () s has a biomial distributio, where - ukow probability of success i oe such test, ad s - the total umber of tests i this series of Beroulli Check coditios for the existece of the cojugate prior distributio; determie its belogig i the class of distributios ad specific parameters of the distributio It is kow that { } x x s x f( x ) = P ξ ( s) = x = C s ( ), x = 0,,,, s There are observed such series Accordig to the formula () we have:

5 America Joural of Applied Mathematics ad Statistics 6 L( x, x,, x ) = f( xi ) xi xi s x = ( ) i Cs xi s xi i ( ) i x = = = C i s, () where x i the umber of successes i i -th series There m = ; T ( x, x,, x) = xi - sufficiet statistic that cofirms the existece of a priori, coupled with L( x, x,, x ) the distributio parameter Now, we will specify the distributio of belogig to a particular class of distributios Defie ( ) = for (0;), the ( x, x,, x) ~ ( ) L( x, x,, x ) xi s xi ~ ( ) (3) The right side of relatios () is (up to a ormalizig factor, idepedet of ) the desity of the beta distributio Г( a b) a b p( + ) ( ) Г( a) Г( b) = (4) with parameters a = xi + ad b = s xi + respectively, where Г( z) - the kow Euler gamma fuctio (see Example a)) Cosequetly, the family of cojugate prior distributios parameter (the probability of "success") observed the biomial distributio of the total populatio belogs to the class of beta - distributio (4) It is kow that the average value ( E ) ad variace ( D ) beta distributio parameters ad expressed i terms a ad b of this distributio by the formulas: a ab E = = 0, D = = a+ b ( a+ b) ( a+ b+ ) Usig these expressios we obtai as the solutios of the system of two equatios relatively a ad b : 0( 0) 0( 0) 0 a 0, b = = 0 0 We defie the posterior distributio, takig ito accout (0) ad the expressio of gamma desity - distributio: x i α β ( x, x,, x ) ~ e e From here, it is see that the posterior distributio of parameter agai obeys the gamma - distributio, but with the followig parameters: = +, = + xi α α β β v) The radom variable with ukow value of the parameter has a uiform distributio o the segmet [ 0; ] Check the coditios of existece of the cojugate priori distributio, to determie its accessories thereof Class of distributios ad the specific values of the parameters of the distributio It is kow that for 0 x 0, f( x ) = 0 for x [ 0; ] From equatio () have: L ( x, x,, x ) =,for x max ( ) = max x i i (5) Therefore, withi the overall view (6) we have: m = ; T ( x, x,, x ) = xmax ( ) - sufficiet statistic that cofirms the existece of a priori, cojugated with L( x, x,, x ) the distributio of the parameter Now will update the belogig of this distributio to a certai class of distributios Takig ito accout (7), from (8) we obtai: ( x, x,, x ) ~ ( ) L( x, x,, x ) ~ for xmax ( ) (6) The right side of the relatios (6) is the desity distributio of the Pareto of the form α αmi for p( ) = α mi + 0 for < mi with parameters α = ad some parameter shift mi x max ( ) (7) Cosequetly, the family of cojugate prior 0; of parameter distributios o the segmet [ ] uiformly distributed radom variable belogs to the Pareto distributio of the form (7) We defie a priori distributio, takig ito accout (5) ad the expressio desity Pareto distributio (7) as a priori distributio we have: p( ; D) = ρα ( ; ; mi ) α mi ( x, x,, x α ) ~ α + for { x x x } max mi ;,,, It follows that the a priori distributio of the parameter is described, as well as a priori, by Pareto law (7), but with the followig parameters: α = α +, mi = mi = max { mi ; x, x,, x }

6 6 America Joural of Applied Mathematics ad Statistics Example A result of ispectio 0 firms it was foud that 6 firms have a average reliability, 3 firm - high reliability ad firm - low reliability A priori reliability probabilities equal 05, 03, 0 respectively Fid a posteriori expectatios of probability reliability of firms I this example, there are three evets ( m = 3 ): firms have a average reliability (the first evet - ω ), high reliability (the secod evet -ω ), low reliability (the third evet - ω 3 ) The probabilities of these evets are radom variables ad mathematical expectatios of probability reliability of firm ( t, t, t 3 ) for the Dirichlet a priori distributio equal: t = 05, t = 03, t 3 = 0 Cosiderig that, = 6, = 3, 3 =, N = 0 ad took s =, the posterior probability of reliability expectatios of firms are equal: t = = 059, 0 + t 3 t = = 03, = = Thus, after receivig the ew statistical iformatio probability of the reliability of firms are radom, but with ew mathematical expectatios obtaied above I this iformatio cofirmed the a priori idea of the probability of firms of high reliability ad chaged the idea of the probabilities of low ad medium of firms reliability O oe had, if we take s = 0, the the a posteriori iformatio be fully determied oly by the available additioal iformatio as aalysis 0 firms The data of coversio i this case do ot deped o the choice of a priori probabilities O the other had, if the parameter s takes large values, the resultig statistics practically ceases to affect the posterior probabilities ad the search for additioal iformatio loses its meaig Bayesia models have several advatages compared with the frequecy models Oe such advatage is that Bayesia models ca i priciple give some results, eve if ot at the sample data This is due to the use of the a priori probability distributio, which i the absece of statistical data does ot chage, ad calculated by Bayes' theorem the posteriori distributio coicides with the a priori 4 Coclusios I decidig Bayesia models allow to carry out additioal experimets to clarify the states of ature I practice, this refiemet is maily by collectig additioal iformatio, ad by performig these experimets Additioal iformatio about the profit ad timely settlemet with the budget allows us to refie the a priori probabilities of states of ature (for reliability) At the same time, the desig of experimets to determie whether it is expediet from the viewpoit of the cost of its implemetatio To do this, you eed to compare the expected additioal icome or beefits that be obtaied by usig additioal iformatio acquired as a result of the experimet, ad the expected cost of the experimet So, for example, with a aalysis of the reliability the firm for iformatio about the profit may demad a audit o which to sped certai meas The umber of alterative courses of actio i such problems icreases as a opportuity to ot oly select oe of the available alteratives, but also determie the feasibility of carryig out experimets or activities for additioal iformatio Refereces [] Tulupyev, AL, Nikoleko SI, Sirotki AV, BAYESIAN NETWORKS: A Probabilistic Logic Approach (Russia), St Petersburg: Nauka, p [] Morris, UT, The sciece of maagemet: A Bayesia approach М: Мir, p [3] Friedrich Schreiber, Probability ad Bayesia Statistics Pleum Press, The Exteded Bayes-Postulate, Its Potetial Effect o Statistical methods ad Some Historical Aspects, New York, 987, pp [4] Jeffreys, Scietific Iferece d ed Cambridge Uiversity Press, 957 [5] De Groot MH Optimal Statistical Decisios: Wiley, 004

Statistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.

Statistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample. Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized