A Primer For Conant & Ashby's Good-Regulator Theorem

Transcription

1 A Pime Fo Conant & Ashby's Good-Regulato Theoem by Daniel L. Scholten

2 Page 2 of 44 Table of Contents Intoduction...3 What the theoem claims...4 Poving the claim...9 The Regulato's Responsiveness to the System...9 The Outcomes Poduced by R's Responses to S...2 Optimal Regulation...5 A Useful and Fundamental Popety of the Entopy Function...8 Expessing H(Z) in tems of P(R S)...20 A Lemma Regading Successful Regulatos...29 The Simplest Optimal Regulato...36 Conclusion: a Rigoous Theoem...4

3 Page 3 of 44 Intoduction Unless you have sufficient confidence in you math sills and especially a basic familiaity with infomation theoy, it can seem a daunting tas to undestand the poof given by Roge C. Conant and W. Ross Ashby (hencefoth C&A) of thei theoem that establishes that evey good egulato of a system must be a model of that system. The pupose of the pesent pape is to tain the eade to accomplish this tas. As to why any eade might want to accomplish this tas I will only say hee that Humanity's inceasing dependence on its aleady huge, still gowing, and inceasingly complicated system of models and epesentations stongly suggests that evey civilized peson eally should have some sot of eniched and high-level undestanding of this fundamental esult fom the System Sciences. It is my belief that this theoem ought to be included in the standad science cuiculum along side such othe basics as the gem-theoy of disease, the sun-centeed view of the sola system and the meaning of symbolic scibbles such as 2+2=4. I have elsewhee exploed this topic in geate detail and in this essay I will focus somewhat naowly on the theoem itself 2. In the pages that follow I will attempt to povide a self-contained exposition of the poof of this Good-Regulato Theoem in a way that equies a minimum of peequisites so that any liteate adult with sufficient motivation and some pio expeience with vey basic pobability theoy, the logaithm function, and pehaps a vey little bit of calculus 3 might follow thei agument. Although this will substantially lengthen the agument (44 pages to explain what amounts to a single page in the oiginal aticle), the hope is that the much longe agument will be easie to undestand. Please notice that I said that I will attempt to povide this. Whethe I succeed will have to be detemined by eades lie you, and if you should detemine that I have fallen shot of this goal, you ae invited to send me you suggestions fo impovement so that futue vesions of this essay will come close to it 4. Let s begin with the authos main obective: Roge C. Conant and W. Ross Ashby, Evey Good Regulato of a System Must be a Model of that System, Intenational Jounal of Systems Science, 970, vol., No. 2, Fo an accessible, non-specialist teatment of the theoem, see The Thee Amibos Good-Regulato Tutoial. A somewhat moe advanced technical analysis can be found in Evey Good Key Must Be A Model Of The Loc It Opens. Both of these esouces ae available fee of chage on the Education Mateials page at 3 The bief passage involving Calculus can be simmed with little loss to oveall undestanding. 4 Please fowad you comments to me at dlscholten@goodegulatopoect.og.

4 Page 4 of 44 What the theoem claims We will begin by taing a close loo at what C&A actually poved by thei theoem. As the title of thei pape poclaims: Evey good egulato of a system must be a model of that system. This assetion uses the tems egulato, system and model, and because each of these can be intepeted in vaious ways we ae going to fist claify what they mean in the context of the Good-Regulato Theoem. Fist of all, the tems system and egulato ae being used hee to efe to dynamic entities, meaning that they can exhibit vaious state-changes. We will efe to these state-changes somewhat colloquially as behavios, so that even a andomly changing system, such as a weathe system, will be descibed as exhibiting behavios. Futhemoe, the egulato in question is such that its behavios can be goal-diected, which is to say that it can execute its behavios in the sevice of some pefeed stateof-affais. Although this does not imply that the egulato must theefoe be a human being (a mechanical device such as a Watt Goveno can be legitimately descibed as goal-diected ) it ust happens to be the case that human beings mae geat examples of such egulatos. Now, it might also be the case that the system in question is goaldiected, but in the cuent context this is not a necessay attibute of the system. The system might be a goal-diected human being, but it might also be no moe goaldiected than the weathe. Anothe point to ecognize is that the system and the egulato ae inteacting. As it concens the cuent context, what this means specifically is that the egulato is attempting to attain a goal and the system eeps doing things that mae it difficult fo the egulato to accomplish this. In this sense, then, the system is eally a system of obstacles and it is the egulato's ob to handle o espond to those obstacles in such a way that the goal is achieved. Also, the Good-Regulato Theoem maes specific efeence to something called a good egulato. In this context, the wod good means quite specifically that the egulato in question is both optimal and maximally simple. It is optimal in the sense that it does the best possible ob of achieving its goal unde the given cicumstances, and it is maximally simple in the sense that it does this best-possible-ob with the least possible amount of effot o expense. Next, the tem model is being used to descibe a epesentational elationship that can exist between some given obect the so-called model and some othe obect the thing being modeled. Hee we have to be somewhat caeful about what this actually means. The tem model has a colloquial intepetation that tends to imply a visual esemblance between the model and what it epesents, but this sot of definition is too naow fo ou puposes. It would exclude, fo example, the type of epesentational activity that goes on, say, wheneve we mae a gocey list. Such a list can hadly be said to have any sot of visual esemblance to the actual items epesented on the list, and yet a gocey list is clealy a epesentation of those items. O conside the type of modeling that is used duing a Monday lunch-hou ecap of

5 Page 5 of 44 Sunday's football game: the peppe shae epesents the quatebac, the etchup bottle epesents a lineman, etc., and yet none of these items beas any sot of visual esemblance to an actual football playe. Although ou definition of model will cetainly allow fo the possibility of such visual esemblance, it will not be a equiement. Instead, the definition we will use is gounded in the mathematical idea of a mapping (a..a. Function) 5. The essence of such a mapping is that it somehow associates all of the vaious component bits and pieces of the thing-modeled to the component bits and pieces of the model. In the cuent context, the model is going to be the egulato, the thing-modeled will be the system and the component bits and pieces will be thei espective behavios. A specific example will mae all of this cleae while intoducing some of the mathematical symbolism we will need to pove the theoem. Fist, let s suppose we ae hoping to egulate some system, call it S 6, which can exhibit, say, six distinct and mutually exclusive behavios. Note that the equiement hee that the behavios be mutually exclusive simply means that if it seems that the system can do two o moe behavios at once pehaps sing a song while standing on its head then such composite behavios need to be teated as sepaate behavios. Thus, a behavio such as singing while standing on it head would be consideed distinct fom eithe meely singing o meely standing on its head. Using some simple set-builde notation we can epesent the complete behavioal epetoie of this system as S = { s, s2, s3, s4, s5, s6. Note also that S is undestood to be the complete behavioal epetoie of the system meaning that it contains evey possible distinct and mutually exclusive behavio that the system could eve execute, even if one of those behavios is some sot of null behavio such as sit and do nothing. If we can ecognize it as something that the system can do it, then it needs to be epesented in S as well. Futhemoe, let's suppose that the only thing we eally undestand about the inne woings of this system is the elative fequencies with which it is executing these behavios. That is, suppose we have a pobability distibution of the fom =,,,,, p s is the pobability p ( S ) { p ( s ) p ( s2 ) p ( s3 ) p ( s4 ) p ( s5 ) p ( s 6 ) whee ( ) that S executes behavio s. Although fo now we won't eally do much with this pobability distibution, when we get to the actual poof of the theoem we ae going to need it and so I want to at least intoduce it hee In thei oiginal pape, Conant and Ashby emphasized the distinction between a system as a thing and the set compised of that thing s possible behavios. They did this by using the symbol S (without italics) to epesent the thing and S (with italics) to epesent that thing s behavioal epetoie, i.e. the set compised of evey behavio that S can pefom. They did something simila with the egulato, using R and R to distinguish between the egulato as a thing and its behavioal epetoie, espectively. Although I agee that this is an impotant distinction to maintain, I will not use two diffeent font styles to maintain it. Instead, I will ely on the eade s good udgment and ability to ecognize the distinction fom contextual clues. Thus, I will wite S o R (with italics) to sometimes epesent the thing (system o egulato, espectively) and sometimes to epesent that thing s behavioal epetoie.

6 Page 6 of 44 Next, let s suppose that off to the side some whee we have built a device that we ae hoping to use to egulate that system a candidate egulato, although to steamline ou discussion we will efe to this device simply as a egulato egadless of whethe it is actually egulating the system. Also, let's suppose that we have built this device to exhibit ust fou distinct and mutually exclusive behavios and similaly to what we did with the system let's call this egulato R and use R =,,, to epesent the complete behavioal epetoie of this egulato. { Now, we have built this egulato as an isolated entity; that is, we can set it unning and it will stat pefoming its vaious behavios in some sot of sequence, but because it is cuently a sepaate entity its behavios will not inteact in any way with the behavios of S. Late on we will conside how to set this egulato up so that it does inteact with the system and in fact egulates that system, but fo now we ae only going to conside how it might be set up to epesent o model that system. As mentioned above, the sot of modeling elationship we ae going to use equies only that the component bits and pieces of the system be associated in some way with the component bits and pieces of the model, whee in ou cuent context these bits and pieces ae undestood to be the behavios of the system and egulato espectively. Now, having set aside fo the time being ou hope of using ou egulato-device to actually egulate the system, and given only ou desie to use it as a model of that system, thee ae actually lots and lots of ways this might be done. Conside the following example: h s s s s s s The diagam shown above is simply a loo-up table that displays the essential details of what is meant by the statement R is a model of S. Reading fom the table, we see that the statement R is a model of S simply means that: Wheneve S does s then R always does 3 ; Wheneve S does s 2 then R always does ; Wheneve S does s 3 then R always does 3 ; Wheneve S does s 4 then R always does 2 ; Wheneve S does s 5 then R always does 3 ; and Wheneve S does s 6 then R always does. We can obseve two othe points about this table. Fist of all, the table has a name h, and second, thee is also an aow that helps us distinguish between the model

7 Page 7 of 44 and the thing-modeled. Notice that this aow points downwad fom the system side of the table towad the egulato side of the table. This is meant to show that in this case it is the egulato that is the model and that it is the system that is the thing being modeled. The standad mathematical shothand used to symbolize this sot of elationship between h, S and R is ust h : S R, which is ead h is a mapping fom the set S to the set R. The above is ust one somewhat abitay example of how ou egulato could be used to epesent o model the system. Hee ae a few othes: f s s s s s s m s s s s s s q s s s s s s Remembe that we ae not yet saying anything about whethe the egulato is actually egulating the system. Maybe it is and maybe it isn't. We will get to that shotly, but at this point we ae only examining this idea of using the egulato as a model of the system. In ode to establish such a epesentational elationship between the egulato and the system we fist need to map the system's behavios to the egulato's behavios, and thee ae lots and lots of ways this might be done. The above mappings ( h, f, m and q ) ae ust fou of these. Anothe impotant point to ecognize about these sots of mappings is that in each case all of the elements in the set S ae associated to an element in the set R, but the opposite is not necessaily tue. In the last example shown ( q : S R ) all of the system's behavios ae being epesented by ust one of the egulato's behavios and the egulato's emaining behavios do no eal epesentational wo at all. This situation coesponds to the type of modeling being done, say, when we use a peppe shae as a model of a quatebac. In that case, all of the complex bits and pieces of the actual quatebac ae mapped on to a single simple attibute of the peppe shae the fact that it is a solid, discete obect that can stand and all of the peppe shae's othe bits and pieces (the peppe, the scew-on lid, the glass body, etc.) do no actual epesentational wo. On the othe hand, the situation illustated by m : S R in which all of the egulato's behavios have been used but because R has fewe behavios than S, some of these do moe epesentational wo than the othes this type of situation coesponds to the use, say, of a toy ca as a model of a eal ca. In that case, all of the obvious component bits and pieces of the toy ca

8 Page 8 of 44 the wheels, windshield, bumpes, etc. have been associated with analogous bits and pieces of the eal ca, but some of these have been used moe than once. Fo example, the simple slab of plastic that uns along the bottom of the toy ca is used to epesent all of the complex bits and pieces that ae beneath the eal ca the muffle, chassis, bea lines, etc. To summaize the above analysis of the tem model: wheneve we have two sets call them A and B and some mapping call it t whee this mapping is fom the elements in A to the elements in B, then we will symbolize this elationship between the thee of them by witing t : A B, and we will also say that B is a model of A. Within the context of ou system and egulato, a moe colloquial way to paaphase all of this is to say that R is a model of S in the sense that R's behavios ae ust S's behavios as seen though some mapping. Equipped with this much moe pecise vocabulay, we ae now eady to examine what the C&A theoem claims which is that wheneve a given egulato is acting as a so-called good-egulato of some given system, then it must also be tue that the egulato is a model of the system in the sense that the egulato's behavios ae ust the system's behavios as seen though some mapping. Anothe way to say this last pat is that wheneve the system executes some given behavio, the egulato always esponds in exactly the same way. Convesely, to the extent that a egulato vaies its esponses to any given system behavio (and thus ceases to be a model of the system), it must also be the case the the egulato is eithe not doing as well as it might, o else it is doing so in an unnecessaily complicated fashion. The autho s pac eveything we ve discussed above into a concise theoem which we can be stated as follows: Theoem : The simplest optimal egulato R of a system S poduces behavios fom R {, 2,, R S { s,, s2 s S = L which ae elated to the behavios in = L by a mapping h : S R. 7 The above is the actual statement that we ae woing towad poving. (Note: the symbols R and S ae ust mathematical shothand fo the numbe of elements in the sets R and S espectively). 7 Conant and Ashby's woding is slightly diffeent, but equivalent.

9 Page 9 of 44 Poving the claim The Regulato's Responsiveness to the System Having claified what we mean by the tems system, model and egulato, we can now tun ou attention to what it means to say that the egulato is esponding to the system. Futhemoe, we will also need some way to epesent the outcomes of these esponses, but fo now we will focus only on R 's esponsiveness to S. Thee ae vaious ways we might intepet what it means to say that R esponds to S but C&A wanted to mae thei esult as geneal as possible and so they chose to use what is nown as a stochastic (pobabilistic) appoach. (This is consistent with the pobability distibution p ( S ) intoduced ealie fo the behavios in S ) That is, they chose not to get bogged down in tying to conside all of the vaious possible paticula mechanisms that might be used to mae R espond to S and instead they chose to use a method that simply specifies fo each possible system behavio a coesponding set of conditional pobabilities ove the entie behavioal epetoie of R. This appoach is ust about as geneal as we could get and applies to any conceivable way to mae R espond to S. It can even be used to descibe situations in which R 's behavio is completely deteminate. No matte what the specific technical details might be, such a method will always allow us to mae statistical statements of the fom, wheneve the system executes s, thee is an x % chance that the egulato will espond by executing i. Thoughout what follows we will epesent such a conditional distibution in one of two ways. Sometimes we will use set-builde notation and epesent it as follows: { p R S = p s : R, s S And sometimes we will use a table notation and epesent it as follows: ( ) p R S s s L s 2 ( ) ( ) L ( S ) ( ) ( ) L ( S ) p s p s p s 2 p s p s p s M M M M M ( ) ( 2 ) L ( ) p s p s p s R R R R S S

10 Page 0 of 44 s In eithe case, fo any egulato behavio, say i R, and any system behavio, say S, the numbe p ( i s ) is the conditional pobability that the egulato will espond by executing behavio i given the condition that the system executes behavio s. Note that in dealing with such conditional distibutions it should always be ept in mind that each numbe in the distibution is between 0 and inclusive and that the sum of the numbes in any column must total. Moe fomally, we will wite: i = 0 p s, fo all R and all s S and R i i p s =, fo all s S i To etun to ou cuent example, since thee ae an infinite numbe of eal numbes between 0 and, then clealy thee ae an infinite numbe of ways we might ceate such a conditional distibution in ode to specify out egulato s esponsiveness to the system. One (somewhat complicated) example would be the following: ( ) p R S s s s s s s What the above distibution tells us, fo example, is that wheneve the system executes behavio s the egulato will neve espond by doing behavio 3 (since p ( 3 s ) = 0 ) but that it will do behavio 4 on oughly half of all such occasions p 4 s =.50 ), and that it will do behavios (since and 2 on oughly 20% and 30% espectively of all such occasions ( p ( s ) =.20 and p s =.30 ). As anothe example, the above table tells us that R will execute in esponse to 85% of the times that the system executes behavio s 4, that R will do 2 and 3 on 0.% and 4.9% of all such occasions espectively, and that it will neve do 4 on such occasions. As a thid example, the above schedule tells us that the R will always do 2

11 Page of 44 in esponse to the system doing s 5 and neve do any of its othe behavios in such a situation. Clealy the above distibution fulfills the two conditions fo any such conditional distibution. That is, each numbe in the distibution is between 0 and inclusive, and if you pic any column and sum the numbes in that column you aive at a total of. The meaning of this latte fact is that the egulato s behavioal epetoie R = {, 2, 3, 4 is eally a complete inventoy of all possible egulato behavios and since it is a tautology to say that the egulato must always be doing something that it can do then the pobability that it does something it can do (given the system has done, say s ) must equal. In othe wods, and since R s behavios ae mutually exclusive, p = { the egulato does something it can do the system has done s p{ the egulato does o 2 o 3 o 4 the system has done s p ( s ) p ( 2 s ) p ( 3 s ) p ( 4 s ) = = Now, the peviously displayed conditional distibution epesents ust one of the infinite numbe of ways we might set up the egulato R so that it is esponsive to S and most of these infinite ways ae athe complicated. But thee is a much simple way we might do this. Fo example, we might use the following distibution: ( ) p R S s s s s s s What this much simple sot of distibution tells us is that wheneve the system does some paticula behavio, the egulato s esponse is always the same. (That should sound familia, fo easons to be explained shotly). Fo example, given that the system does, say s, then the pobability that the egulato does 3 is (that is, p ( 3 s ) = ), meaning that it is cetain that the egulato does 3 wheneve the system does s and that the egulato will neve do any othe behavio in its epetoie

12 Page 2 of 44 when the system does p s = p 2 s = p 4 s = 0 ). Of couse, the egulato s esponse might change when the system s behavio changes. Fo example, accoding to the above distibution, wheneve the system switches to behavio s 4 the egulato will always switch to doing 2 but that is the only time the egulato might change its esponse. As long as the system does the same behavio and wheneve it does that same paticula behavio, the egulato s esponse is always the same esponse to that paticula system behavio. s (since The ey thing to notice hee is that this is pecisely the sot of situation that is descibed by the tem mapping. Notice that the esult of the above vey simple sot of pobability distibution is that each one of the system's bits and pieces (i.e behavios) is mapped to exactly one of the egulato's bits and pieces that is, the paticula egulato behavio that is pefomed with pobability in esponse to the given system behavio. In fact, this paticula mapping is exactly the same one we consideed ealie and which we epesented as the following table: h s s s s s s You should tae a moment to convince youself that the above table and the pevious conditional pobability distibution ae eally ust two diffeent ways to specify the exact same mapping h : S R. Although the conditional pobability distibution is suely the moe complicated of the two, we ae going to need this exta complication in ode to pove the Good-Regulato Theoem. The Outcomes Poduced by R's Responses to S So, that is how we will epesent R s esponsiveness to S, via some conditional distibution p ( R S ), which, in the geneal case will not necessaily specify a mapping fom S to R, but which at least could do so (and thus mae R into a model of S ). Now it is time to conside what actually happens wheneve R esponds to S, that is, the actual outcomes that aise fom these esponses. The eason these outcomes ae impotant, of couse, is that they fom the vey substance of the egulato's goal. In ode to epesent the outcomes that aise wheneve R esponds to S, C&A mae use of a diffeent sot of mapping. As with any mapping, this one will also have a name, the Gee lette, ponounced sigh and involve two sets, but the fist of these sets is actually a ind of mixtue of the sets S and R. This mixtue is called the coss-poduct of R and S, and is symbolized as follows:

13 The coss-poduct of R and S = R S = {, s : R, s S Page 3 of 44 The elements in this set ae nown as odeed pais and each odeed pai in that set consists of a single element fom the set R along with a single element fom the set S. Fo example, if the element fom R is i and the element fom S is s, then we can symbolize the odeed pai that consists of these two elements as, s i. The cosspoduct of R and S, then, is the set that consists of evey possible such odeed pai. That descibes the fist set involved in the mapping we ae calling. The second set is ust the set of evey possible outcome that could aise fom some combination of an S behavio and an R behavio. We e teating the most geneal case hee so we won t be concened with any paticula details of such outcomes, but we will intoduce a thid set Z { z, z2,, z Z = K to epesent these possible outcomes. Thus, the pupose of the mapping is to lin each paticula combination of a egulato behavio and a system behavio, that is, each odeed pai, s R S, to a unique esult o outcome in the set Z. Using the standad shothand intoduced above we can epesent this mapping as : R S Z. (Of couse, based on ou ealie discussion of models, the existence of this mapping means that we can also say that Z is a model of R S, but this paticula model is not the one we ae eally concened with hee and so we will ust ignoe this option. As it concens ou pesent discussion of the outcomes that aise fom the egulato's esponses to the system, we ae only inteested in the actual mapping : R S Z.) Now, if we happen to now, fo example that the paticula odeed pai i, s R S maps unde to the paticula esult z Z, then we could epesent this paticula fact as (, s ) i = z, o pehaps the moe steamlined ( i, s ) z as C&A do, which endes the angle bacets implicit. =, To illustate all of this with a moe concete example, let s use the sets R,,,, S = s, s, s, s, s and fo the set of possible esults we ll use = { and { {,,,,,,, Z = z z z z z z z z. The sizes of the sets R and S in this example ae a little diffeent fom the ones we used ealie, but these diffeences ae supeficial and the only eason I ve made them diffeent is to illustate that the diffeences don t eally matte. One thing we should ecognize about these sots of examples is that they ae not meant to imply any estictions on the sizes of the sets R, S o Z, (epesented by R, S and Z, espectively) eithe in an absolute sense o elative to each othe, and in pactice any of these, in fact, may be infinite. Thus, although in this example we have that Z = 8 > 5 = S = R this is eally ust a matte of haphazad

14 Page 4 of 44 convenience. In the geneal case we might have any of the following: Z R S, S Z R, S R Z, R S Z o R Z S. Z S R, One point to ecognize hee is that thee ae an infinite numbe of ways that might map elements in R S to elements in Z. The paticula one we will use fo ou cuent discussion is aanged in the following table: s s s s s z z z z z z z z z z z z z z z z z z z z z z z z z The above table, called a payoff matix, is to be ead as you would a multiplication table. That is, if you want to now, fo example, the paticula element of Z to which maps the odeed pai, s R S, then you loo in the cell of the i table that lies at the intesection of the table s i ow and the s column. Thus, the table specifies that (, s ) = z, ( 4, s5 ) = z4 and (, s ) also indicates that (, s ) (, s ) (, s ) z = z. Note that the table = = = which is meant to illustate the moe geneal possibility that the same outcome (e.g. z 6 ) can be obtained via in esponse to vaious diffeent combinations of the elements in R S (e.g. fo z 6 these would be the odeed pais 2, s, 3, s 4 and 5, s 2 ). A elated point to notice is that some of the columns in paticula the columns fo s 2 and s 4 contain epeated occuences of the same outcome. Fo example, the s 4 column shows the outcome z in both the ow fo 2 as well as the ow fo 4. This possible chaacteistic fo such a table will play an impotant ole late on in the agument. Just as a point of compaison, othe possible mappings example sets R, S and Z would be any of the following: : R S Z fo ou

15 Page 5 of 44 s s2 s3 s4 s5 s s2 s3 s4 s5 s s2 s3 s4 s5 z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z z And as an illustation of examples involving diffeent sets R, S and Z conside the following: s s s s s s s z z z z z z z z z z z z z z z z z z z z z and s s s s z z z z 5 9 z z z z z z z z z z z z z z z z z z z z Optimal Regulation Once we have specified a conditional distibution p ( R S ) and the mapping : R S Z, then we have eveything that we need to now both about the egulato's esponsiveness to the system as well as what actually happens as a esult of that esponsiveness. The next question we have to addess concens what it means to say that a egulato is optimal, o, that it is doing the best-possible ob unde the given cicumstances. We can also efe to this as successful egulation. In ode to define what it means to say that a egulato behaves optimally, let s stat by consideing an actual egulato. Fo example, suppose we want to puchase a themostat in ode to egulate oom tempeatue. How could we tell if a themostat is a good themostat? Clealy, a themostat is successful when it behaves in such a way as to eep the oom tempeatue constant, o as close to constant as possible, especially when the outside tempeatues ae fluctuating. Genealizing fom this example we might conclude that any egulato can be consideed successful if it behaves in such a way as to achieve as much constancy o as little change as possible in the set of outcomes it poduces. Anothe way to say this is that a successful egulato should educe as much as possible the unpedictability in the set of outcomes it poduces. But hold on. Suppose we puchase a themostat and discove once it is installed that it does a fantastic ob of maintaining a constant oom tempeatue, but that the only

16 Page 6 of 44 constant oom tempeatue it maintains is 350 F! Should we still conside this egulato successful? Suppose futhemoe that it does such a geat ob of maintaining the oom tempeatue at 350 F that it could do so even if the oom wee placed on the suface of the planet Mecuy whee the outside tempeatues ange fom a low of -300 F to a high of 800 F. Now how should we evaluate the success of this themostat? On the one hand, 350 F is a lousy oom tempeatue so we might conclude that the egulato in question is doing a lousy ob. On the othe hand, any device that could maintain a constant oom tempeatue in the face of such exteme outside tempeatue vaiation as is found on the suface of Mecuy is a petty amazing device indeed egadless of what that constant tempeatue might be. Futhemoe, suppose we actually needed to maintain such a constant 350 F tempeatue unde such exteme conditions. That is, suppose we wanted to open, say, a high volume cupcae factoy on the suface of Mecuy, whee we needed to bae cupcaes at 350 F aound the cloc all yea long. In that albeit bizae situation such a egulato would be ust what the docto odeed. Unde such cicumstances, such a egulato would clealy be successful. The point of this example is to illustate that egulato success can be defined in at least a couple of ways. On the one hand, we might say that a egulato is successful as long as it can minimize changes in the outcomes it poduces, egadless of what those outcomes might be. Such a definition would always count as successful such egulatos as the above themostat. On the othe hand, we might add the additional equiement that a egulato should be able to eceive some use's abitay tempeatue equest 350 F, 25 F, 8 F, etc. and then adust itself so as to maintain that equested tempeatue. Such a definition would exclude the above themostat in some situations (if we wee planning to use it, say, to egulate living oom tempeatue) and it would include it in othes (if we wanted to use it to maintain tempeatue in a cupcae oven on Mecuy). Pesumable because the fist appoach is moe geneal, and because the second appoach depends on the somewhat abitay equiements of the context in which the egulato is actually used, Conant and Ashby use the fist appoach. That is, they define egulato success stictly in tems of stability the minimization of the changes in the outcomes poduced by the egulato's esponses to the system. If it tuns out that a given egulato poduces a constant (o elatively constant) set of outcomes that happen to be undesiable in one context, well, then that ust means we have to find the ight context fo the egulato, but we will still count it as a good egulato. Next we need a way to measue the extent to which the egulato is able to achieve such a set of (elatively) constant outcomes. Now, if the outcomes ae associated with some numeic vaiable, such as tempeatue in the case of a themostat, then the standad measuing tool would be the statistical vaiance which is defined as the expected value of the squaed diffeences between the outcomes obtained and the expected value of those outcomes. Fomally, letting X epesent the numeic vaiable

17 Page 7 of 44 associated with the outcome, and letting E ( X ) epesent the expected value of X, then the vaiance of X is defined as, 2 = Va X E X E X But this appoach equies that we have some numeic vaiable we can measue and Conant and Ashby wanted to teat those cases as well as cases in which no such numeic vaiable was available. In ode to accomplish this they use a device fom Infomation Theoy called the Shannon Entopy Function which is basically a measue of vaiation that depends only on the pobability distibution that govens whateve paticula pocess whose vaiation we ae tying to measue. In the cuent context the pocess that concens us involves the occuences of the vaious outcomes in Z that esult fom combining egulato behavios with system behavios and so, letting p z epesent the pobability that some paticula esult z Z is obtained, the pobability distibution of inteest is = ( ), ( 2 ), K, ( Z ) { p Z p z p z p z and the entopy function fo the set Z of outcomes is defined as follows: Z log H Z p z p z = Thus, C&A define an optimal egulato as one that esponds to the system so as to mae the entopy H ( Z ) as small as possible, given R, S, Z, p ( S ) and : R S Z. Notice that in that definition the only vaiable not assumed given is the conditional distibution p ( R S ) which specifies the egulato s esponsiveness to the system. The eason fo this is that, in ou pesent context, the whole point of egulato design is the specification of this conditional pobability distibution. That is, we ae assuming that we, as egulato designes, have total contol ove choosing this distibution in ou seach fo an optimal egulato and also that this is eally the only thing we can contol. In othe wods, we ae assuming that someone has pluned down onto ou wobench some system with a behavioal epetoie S and an associated pobability distibution p ( S ), along with some egulato with a behavioal epetoie R, a set of possible esults Z, and a mapping : R S Z, and that we have been ased to find a way to mae R esponsive to S that is, to design a p R S in such a way as to minimize the associated value conditional distibution of the entopy function H ( Z ). Of couse, this aises the question as to how we ae supposed to accomplish this. We will etun to this question shotly.

18 Page 8 of 44 Actually, C&A mae an additional implicit assumption which I want to go ahead and mae explicit. It s a small but impotant detail, and I want to get it out of the way. This assumption is that the system s behavioal epetoie only contains behavios that the system actually might pefom. Anothe way to say this is that S is such that evey pobability in p ( S ) is geate than zeo. This is easy enough to do. If we stat off with some vesion of S that contains a behavio, say s α, that cannot occu, that is, which is such that p ( s α ) = 0, then we can ust ceate a new vesion of S fom which s α has been excluded. Since s α could neve occu anyway, its exclusion fom S has no impact beyond maing evey pobability in p ( S ) geate than zeo and this is necessay if we want to define p ( R S ) without having to esot to any special notational gymnastics, and that s why I want to mae this assumption explicit. A Useful and Fundamental Popety of the Entopy Function Geneally speaing, the entopy function has a numbe of popeties that mae it a useful measue of unpedictability. Fist of all, fo an abitay pocess with event epetoie X { x, x2,, x X = K the entopy function H X can tae on values that ange between an absolute minimum of H ( X ) = 0 and an absolute maximum of H ( X ) = log X. When H ( X ) = 0 then the pocess is said to be completely pedictable and when H ( X ) log X unpedictable. A situation in which H ( X ) two extemes. = the pocess is said to be completely 0 < < log X is somewhee between these The entopy function has anothe attibute that we will need shotly. The authos descibe this chaacteistic of the entopy function as a useful and fundamental popety which is that if we pic any two pobabilities used to calculate the entopy, and we then incease the imbalance between those two pobabilities, that is, if we mae the bigge one even bigge and the smalle one even smalle, and then if we ecalculate the entopy using these new pobabilities, then the ecalculated entopy will be smalle than the oiginal entopy. I am going to use a little basic calculus now to pove that this popety holds, but if you aen't comfotable with calculus you can ust sip this next pat and tae it on faith that this tuly is a popety of the entopy function. If you ae comfotable with calculus, the agument poceeds as follows.

19 Page 9 of 44 Fist of all we stat with any set of numbes, say p, p2,..., p n, whee each is assumed to be between 0 and inclusive, and then we choose any two of them, say and p b, whee we can assume, without loss of geneality, that pa pb (othewise we ust e-label them). Then we choose any positive numbe δ, whee 0 < δ <, such that pa + δ > pa pb > pb δ 0 (which actually implies that 0 < δ < pb ). Then the claim we wish to pove is the following: (,,..., + δ,..., δ,... ) < (,,...,,...,,... ) H p p p p p H p p p p p 2 a b n 2 a b n In ode to pove this esult, we hold constant the p, p2,..., p n and define the H δ, whee 0 δ < pb, following function ( δ ) = (,,..., + δ,..., δ,... ) H H p p p p p 2 a b n = ( + δ ) ( + δ ) ( δ ) ( δ ) p log p p log p p log p a a b b i i i =, i a, i b n p a Notice that when δ = 0, this function H ( δ ) is equal to the oiginal entopy of ( 0 ) = (,,..., + 0,..., 0,... ) = (,,...,,...,,... ) H H p p p p p H p p p p p 2 a b n 2 a b n Next, still holding constant the, 2,..., n espect to δ : p p p, we tae the deivative of H δ with ( pa δ ) ( pb δ ) dh + = log ( pa + δ ) + log b dδ p + δ p δ a ( p δ ) ( p δ ) = log log a b b ( p δ ) = log ( pb δ ) ( p + δ ) a dh Now we obseve that < 0 fo any δ such that 0 < δ < pb. How do we now dδ this? Well, fist of all, emembe that we assumed that pa pb and so 0 < δ < pb

20 Page 20 of 44 pb δ implies that pa + δ > pb δ and thus that pa + δ < and since the logaithm function is stictly inceasing ove its entie domain we can conclude that dh pb δ dh = log < log = 0. The fact that < 0 when 0 < δ < pb means that the dδ pa + δ dδ function H is stictly deceasing on the inteval 0 < δ < p which means that b ( δ ) = (, 2,..., a + δ,..., b δ,... n ) < H ( p, p2,..., pa,..., pb,... pn ) = H ( p, p,..., p + 0,..., p 0,... p ) H H p p p p p 2 a b n = H (0). The impotant thing to notice in all of that is the following fact: (,,..., + δ,..., δ,... ) < (,,...,,...,,... ) H p p p p p H p p p p p 2 a b n 2 a b n Assuming always, of couse, that 0 < δ < pb and that pa pb, as stated ealie, meaning, as claimed, that if we incease the imbalance between any two pobabilities, we cause a decease in the entopy. Expessing H(Z) in tems of P(R S) Let's get bac to the question we ased a few pages ago. That is, how ae we supposed to find a conditional distibution p ( R S ) that will minimize H ( Z ), given the vaious above mentioned assumptions? Well, the fist thing we need in ode to answe this question is some way of elating the entopy function to the conditional distibution p ( R S ). We will now deive this elationship, but in ode to guide ou deivation and to mae it a bit moe concete we will use the paticula example we p S and have aleady been using. That is, we will use the following fo R, S, Z, : R S Z : R = {,,,, {,,,, S = s s s s s

21 Page 2 of 44 {,,,,,,, Z = z z z z z z z z { =,,,, p S p s p s p s p s p s s s s s s z z z z z z z z z z z z z z z z z z z z z z z z z Now, as mentioned ealie on, once any egulato has been built to exhibit the vaious behavios in R, be it an optimal egulato o othewise, the act of setting it up to espond to that system coesponds to the specification of the conditional p R S which we can epoduce in its geneal fom as it pobability distibution applies to ou example as follows: ( ) p R S s s s s s ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s Remembe that each enty ( i ) p s in the table epesents a numbe that is undestood to be the conditional pobability that the egulato esponds to an occuence of s S with the esponse i R. Remembe also that because these ae pobabilities, each must be a numbe between zeo and one, and because they ae conditional pobabilities (each being conditioned on a paticula column) we equie that the sum of the numbes in any given column must equal exactly one. Moe fomally, we can wite: R i = ( i ) 0 p s, fo each i =,2, K, R, and =,2, K, S ; and ( i ) ( ) ( 2 ) ( R ) p s = p s + p s + L + p s =, fo each =,2, K, S. Of couse, fo ou paticula example we have that S = 5 and R = 5.

22 Page 22 of 44 Now, as it concens ou paticula example, one of the infinite ways we might assign actual numbes to this distibution is as we did ealie: ( ) p R S s s s s s One impotant popety to notice about such conditional pobability distibutions that we will mae use of a little late on is that as long as we espect the two basic conditions (that each numbe in the table is between zeo and one and that the numbes in any column sum to one) then we ae fee to tae any valid such distibution p R S and play aound with any one of its columns in any way we lie to ceate a diffeent distibution, say p ( R S ), that is equally valid. Using ou example above, we can, fo example, change its thid column as shown hee: p ( R S) s s s s s And the esulting distibution, which I ve called p ( R S ), is equally valid, meaning that it still espects the two basic conditions. The point hee is that wheneve we change a column in such a table, we only have to woy about the numbes in that paticula column. Anothe way to say this is that each column is independent of the othe columns. Now, with the distibution p ( S ) assumed given, the specification of the conditional distibution p ( R S ) detemines the oint distibution p ( R, S ), since fo any s S and i R we have that the pobability that both of these occu togethe is p( i, s ) = p( s ) p( i s ) fo each i =, 2, K, R, and =,2, K, S. Fo ou paticula example, we can epesent this oint pobability distibution fo R and S in its most geneal fom as follows:

23 Page 23 of 44 (, ) p R S s s s s s (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) (, ) p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s p s But the specification of p ( R S ) also detemines the pobability distibution {,,, 2 Z p Z p z p z p z = K and thus the entopy function fo H Z. How so? To see this, let s conside ou example. Tae anothe loo at the table fo ou example mapping : R S Z, which I ll epoduce hee: s s s s s z z z z z z z z z z z z z z z z z z z z z z z z z Notice that in this table, the element z 6 appeas thee times; once each in the columns s, s 2 and s 4. That is, ( 2, s) = ( 3, s4) = ( 5, s2) = z6. Given this infomation along with the pobabilities in the oint distibution p ( R, S ), we can calculate p( z 6), the pobability of the outcome z 6. In ode to calculate p( z 6) we fist ecognize that each of the s S ae mutually exclusive as ae each of the i R which means that each of the events { 2, s, { 3, s 4 and { 5, s 2 ae also mutually exclusive. Futhemoe, the event { z 6 occus if and only if any one of the mutually exclusive events {, s, {, s o {, s occus. And this means that we can calculate p( z 6) as the simple sum of each of the pobabilities p( 2, s ), p( 3, s 4) and p(, s ). That is, 5 2 p( z ) = p(, s ) + p(, s ) + p(, s )

24 Page 24 of 44 Reasoning in the same way fo each of the elements of Z yields the following fo the distibution p ( Z ) : p Z = ( ) = ( ) + ( 2 4 ) + ( 4 4 ) + ( 5 5 ) ( 2 ) = ( 2 ) + ( 2 5 ) + ( 4 2 ) + ( 4 3 ) ( 3 ) = (, 3 ) + ( 3, ) + ( 5, ), ( 4 ) = (, 4 ) + ( 4, 5 ), ( 5 ) = (, 5 ) + ( 3, 3 ), ( 6 ) = ( 2, ) + ( 3, 4 ) + ( 5, 2 ), ( 7 ) = ( 2 2 ) + ( 3 2 ) + ( 3 5 ) + ( 5 3 ) = (, ) + (, s ) + p (, s ) p z p, s p, s p, s p, s, p z p, s p, s p, s p, s, p z p s p s p s p z p s p s p z p s p s p z p s p s p s p z p, s p, s p, s p, s, p z8 p 2 s3 p Now, equipped with this pobability distibution fo the elements of Z, we can now calculate the entopy of Z fo ou paticula example as follows: 8 = log = log L log H Z p z p z p z p z p z p z = 8 8 p (, s ) p ( 2, s4 ) p ( 4, s4 ) p ( 5, s5 ) log p (, s ) p ( 2, s4 ) p ( 4, s4 ) p ( 5, s5 ) p (, s2 ) p ( 2, s5 ) p ( 4, s2 ) p ( 4, s3 ) log p (, s2 ) p ( 2, s5 ) p ( 4, s2 ) p ( 4, s3 ) p (, s3 ) p ( 3, s ) p ( 5, s ) log p (, s3 ) + p ( 3, s ) + p ( 5, s ) p (, s4 ) p ( 4, s5 ) log p (, s4 ) p ( 4, s5 ) p (, s5 ) p ( 3, s3 ) log p (, s5 ) p ( 3, s3 ) p ( 2, s ) p ( 3, s4 ) p ( 5, s2 ) log p ( 2, s ) p ( 3, s4 ) p ( 5, s2 ) p ( 2, s2 ) p ( 3, s2 ) p ( 3, s5 ) p ( 5, s3 ) log p ( 2, s2 ) + p ( 3, s2 ) + p ( 3, s5 ) + p ( 5, s3 ) p (, s ) p (, s ) p (, s ) log p (, s ) p (, s ) p (, s ) = It will be useful late to be able to wite these equations out fo the geneal case. p, s will find thei way into The poblem is that we cannot pedict which of the ( i )

25 Page 25 of 44 the summation fo a given p ( z ) and so we cannot use the standad index method to epesent the summation. That is, in the geneal case, we cannot wite anything lie the following: M N ( ) = p ( i, s ) p z = i = Howeve, one way aound this is as follows: p (, s ) p z = ( z ) Whee to z z is the set of all and only those odeed pais, s R S that map Z unde. Moe fomally, ( z ), s R S : (, s ) = z. { Futhemoe, if want to indicate that we ae only summing the oint pobabilities fo, say, the paticula column s, we can use the following notational device: ( ) p (, s ) p z = s ( z ) Whee ( z ) to z s is the set of all and only those odeed pais, Z unde, fo some paticula s {, : (, ) s s R S that map S. Moe fomally, z s R S s = z. Note that fo any two distinct s, s S, the sets s ( z ) and s ( z ) h ove S is ust the set ae mutually exclusive. Also, the union of all such sets. Putting this fomally, we have z h S s s2 s S U s = ( z ) = ( z ) ( z ) L ( z ) = ( z ) Using this latte notation we can now wite p ( z ) as a double sum using a hybid indexing style as follows:

26 Page 26 of 44 S ( ) = p (, s ) p z = s ( z ) This notation allows us to wite p( z ) in tems of the conditional distibution p ( R S ) as follows: S = = ( ) p ( s ) ( z ) p z p, s p, s ( ) = s z = s ( z ) p s = S = p s s S p s Let s tae anothe loo at that last expession. In paticula, let s focus on the expession used in the inne sum: s ( z ) ( ) p s The fist thing to ealize about this expession is that it is the conditional pobability that the egulato poduces the outcome z in esponse to s S. That is, ( z ) s ( ) ( ) p s p z s = Secondly, in the geneal case, and fo evey s zeo and one (inclusively). That is, S, this quantity is always between ( z ) ( ) ( ) 0 p s = p z s s

27 Thidly, fo any paticula s occuences of z in the occuence of z in the would have, fo that paticula s, Page 27 of 44 S, it may be the case eithe that thee ae no s column in which case ( z ) s column ( ) ( z ) = o that fo evey s p s = 0. If eithe of these is tue then we ( ) ( ) p s = p z s = 0 s And fouth, again fo any paticula s outcome in the have, fo that paticula s, s column fo which ( ) ( z ) S, it may be the case that z is the only p s > 0. When this is tue then we would ( ) ( ) p s = p z s = s To illustate all of this with ou paticula example we would have the following sets, fo =,2, K, Z. z fo ( z ) = {, s, 2, s4, 4, s4, 5, s5 ( z2 ) = {, s2, 2, s5, 4, s2, 4, s3 ( z3 ) = {, s3, 3, s, 5, s ( z4 ) = {, s4, 4, s5 ( z5 ) = {, s5, 3, s3 ( z6 ) = { 2, s, 3, s4, 5, s2 ( z7 ) = { 2, s2, 3, s2, 3, s5, 5, s3 ( z8 ) = { 2, s3, 4, s, 5, s4 And as fo the ( z ), fo =,2, K, Z we have: s