Variety and organization (long post is...)
Feb. 2nd, 2011 06:49 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
davvElsewhere, I referred to how useful I've found variety in explaining why organizations behave the way they do. In particular, I thought a bit about why nations with large states tend to encourage the people to be uniform, through a sort of "equality by uniformity". And then I thought about variety, and just by turning the problem around, looking at it from the right angle, it became simple... and it should show what it is I mean.
To make sense, I'll have to show what I mean by "variety". Variety is the number of possible states of a system. A ball resting on a table has only one state, where it is resting on the table. A heater may have more than one state: it can be on, off, or somewhere in between. If the only choices are full blast or nothing at all, then it has a variety of 2.
I also have to show what is meant by control, in the technical sense. A controller or regulator limits the variety of the system it is controlling. A room without a thermostat-controlled heater has a wide range of possible temperatures (depending on the temperature outside), but one where the temperature is being regulated has a much more narrow range.
The controller does this by observing and adjusting towards the desired outcome, then observing again, adjusting further, and so on. This is feedback.
So, how does that relate to organizations and governments? There's one more thing to know: in order to control a system, the controller must have at least as much variety as the system[1]. For every move that may take the system out of the desired state, the controller has to have some way of pushing the system back into the desired state. If the controller does not, it will only partially control the system, or fail entirely. That is requisite variety: "variety absorbs variety", or "only variety in R can force down variety in D".
If we take the traditional ideologies of left and right at their face value, then it appears that the distinction, when taken to the extreme is as follows:
The "regulator" in this case is usually the state. However, the state is organized in a bureaucratic manner, which means it has low variety. Unless the state is a police state, it is limited in what actions it can take, and even for those actions it can take, the inefficiency of the bureaucracy means that the "countermove" is only partially successful in limiting the variety of, say, the economy.
Since the state has low variety, the ideologies above are polar. If the regulator is to permit a high variety in society, it must accept a weakening of its ability to regulate or control. On the other hand, if the regulator is to be successful at what it aims to do, which is to limit unfortunate outcomes, it must somehow manage to have society lower its own variety because the state cannot directly handle its task otherwise - it simply does not have the capacity to do so.
Out of that observation, the tendency of left-leaning central governments to desire uniformity becomes easy to see. The state cannot directly control the parameters it wants (economic ones, for instance), and so will promote what I call a type-2/type-3 or majoritarian concept of equality: that everybody is equal if they're uniform. If the people accepts it, that would greatly reduce the variety of society, potentially bringing control within the grasp of the state[2].
One would also assume, based on the above, that right-leaning governments would forgo regulation altogether. That is certainly a goal of its extreme position, but "right" seldom goes as far as to advocate a free-for-all anarchy, for that would destroy freedom. Even the right wants some regulation, and practically speaking, the economic powers (corporations and private organizations) that tend to benefit from a right position will advocate that they, rather than the state, perform the task of a controller, but for their own purposes.
Even though I didn't specifically intend to reach the conclusion when I first wrote this (in a text file of my own), it's surprising how easily the conclusion follows: if the state doesn't have enough variety to regulate well - which it doesn't, being bureaucratic - it will encourage the system to become more well-behaved so it can.
As for me, I think that if left to its own, the system, at least the economic part of it, is unstable. If there is minimal economic regulation, some entity (a company, most likely) will eventually gain enough power to rewrite the rules. At that point, it can decide to be the new state, and the control it then exercises will not be for the good of the people. Pretty much the same thing applies in the absence of any order at all, except the entities are not only economic, they make use of physical force as well. The different mafia gangs will compete until someone ends up on top, at which point the game is over.
The alternative, though, is not very good either. It is better than being in a cyberpunk future, but if the state cannot regulate directly, it will keep trying to simplify the people so that they can be regulated. What is the answer? I think, and this is my opinion, of course, that the regulator must be set up so that it has the requisite variety - not by making the state a police state, but by design of the organization, decentralization, and so on. A recursive design would help, but it must not become too bogged down in sending messages up and down a hierarchy. With the power granted by the high variety in the regulator, the regulator itself (the "state", or more broadly, the group of organizations handling governing functions) has to be anchored by the wishes of the people. Ideally, it should be radically transparent, free of hierarchy, and incorporate ordinary people so that it will resist corruption for as long as possible.
[1] Generally, it must have as much variety as it's trying to reduce in the system being regulated. A bang-bang (on-off) thermostat can limit temperature in a room to a certain range by using some variety, but a PID controller can make use of the heater's ability to be set at other levels than full blast and nothing at all to regulate to a much tighter range.
[2] The more centralized the state, the more variety it has to deal with, and the greater this effect would be. The effect is the same thing that makes micromanagement so impractical. If a center tries to coordinate everything, even if that center is somehow perfectly honest and not corrupted by power, it soon finds itself overextended and quality suffers.
To make sense, I'll have to show what I mean by "variety". Variety is the number of possible states of a system. A ball resting on a table has only one state, where it is resting on the table. A heater may have more than one state: it can be on, off, or somewhere in between. If the only choices are full blast or nothing at all, then it has a variety of 2.
I also have to show what is meant by control, in the technical sense. A controller or regulator limits the variety of the system it is controlling. A room without a thermostat-controlled heater has a wide range of possible temperatures (depending on the temperature outside), but one where the temperature is being regulated has a much more narrow range.
The controller does this by observing and adjusting towards the desired outcome, then observing again, adjusting further, and so on. This is feedback.
So, how does that relate to organizations and governments? There's one more thing to know: in order to control a system, the controller must have at least as much variety as the system[1]. For every move that may take the system out of the desired state, the controller has to have some way of pushing the system back into the desired state. If the controller does not, it will only partially control the system, or fail entirely. That is requisite variety: "variety absorbs variety", or "only variety in R can force down variety in D".
If we take the traditional ideologies of left and right at their face value, then it appears that the distinction, when taken to the extreme is as follows:
- The right ideology thinks that society should be like a cold-blooded animal: to have no regulator, but instead simply go with the dynamics of the system itself. Therefore, it's better to have a free society than to control it, even if the latter could give a better foundation.
- The left ideology thinks that society should be "warm-blooded" because the evolution of society, left to its own premises, will result in very bad outcomes (extreme wealth inequality followed by revolt and collapse). Therefore it's better to have a reasonable foundation to keep the system from going out of spec, and if that requires a regulator, so be it
The "regulator" in this case is usually the state. However, the state is organized in a bureaucratic manner, which means it has low variety. Unless the state is a police state, it is limited in what actions it can take, and even for those actions it can take, the inefficiency of the bureaucracy means that the "countermove" is only partially successful in limiting the variety of, say, the economy.
Since the state has low variety, the ideologies above are polar. If the regulator is to permit a high variety in society, it must accept a weakening of its ability to regulate or control. On the other hand, if the regulator is to be successful at what it aims to do, which is to limit unfortunate outcomes, it must somehow manage to have society lower its own variety because the state cannot directly handle its task otherwise - it simply does not have the capacity to do so.
Out of that observation, the tendency of left-leaning central governments to desire uniformity becomes easy to see. The state cannot directly control the parameters it wants (economic ones, for instance), and so will promote what I call a type-2/type-3 or majoritarian concept of equality: that everybody is equal if they're uniform. If the people accepts it, that would greatly reduce the variety of society, potentially bringing control within the grasp of the state[2].
One would also assume, based on the above, that right-leaning governments would forgo regulation altogether. That is certainly a goal of its extreme position, but "right" seldom goes as far as to advocate a free-for-all anarchy, for that would destroy freedom. Even the right wants some regulation, and practically speaking, the economic powers (corporations and private organizations) that tend to benefit from a right position will advocate that they, rather than the state, perform the task of a controller, but for their own purposes.
Even though I didn't specifically intend to reach the conclusion when I first wrote this (in a text file of my own), it's surprising how easily the conclusion follows: if the state doesn't have enough variety to regulate well - which it doesn't, being bureaucratic - it will encourage the system to become more well-behaved so it can.
As for me, I think that if left to its own, the system, at least the economic part of it, is unstable. If there is minimal economic regulation, some entity (a company, most likely) will eventually gain enough power to rewrite the rules. At that point, it can decide to be the new state, and the control it then exercises will not be for the good of the people. Pretty much the same thing applies in the absence of any order at all, except the entities are not only economic, they make use of physical force as well. The different mafia gangs will compete until someone ends up on top, at which point the game is over.
The alternative, though, is not very good either. It is better than being in a cyberpunk future, but if the state cannot regulate directly, it will keep trying to simplify the people so that they can be regulated. What is the answer? I think, and this is my opinion, of course, that the regulator must be set up so that it has the requisite variety - not by making the state a police state, but by design of the organization, decentralization, and so on. A recursive design would help, but it must not become too bogged down in sending messages up and down a hierarchy. With the power granted by the high variety in the regulator, the regulator itself (the "state", or more broadly, the group of organizations handling governing functions) has to be anchored by the wishes of the people. Ideally, it should be radically transparent, free of hierarchy, and incorporate ordinary people so that it will resist corruption for as long as possible.
[1] Generally, it must have as much variety as it's trying to reduce in the system being regulated. A bang-bang (on-off) thermostat can limit temperature in a room to a certain range by using some variety, but a PID controller can make use of the heater's ability to be set at other levels than full blast and nothing at all to regulate to a much tighter range.
[2] The more centralized the state, the more variety it has to deal with, and the greater this effect would be. The effect is the same thing that makes micromanagement so impractical. If a center tries to coordinate everything, even if that center is somehow perfectly honest and not corrupted by power, it soon finds itself overextended and quality suffers.
no subject
Date: 2011-02-03 07:28 pm (UTC)no subject
Date: 2011-02-04 03:48 pm (UTC)A state might have a large body that makes general decisions and appoints smaller bodies to implement those decisions, recalling the bodies if they do badly. This sounds like a hierarchy, but the "top" (where power resides) is made up of those who are affected by the decisions of the implementers. A parliamentary democracy is somewhat like this, because the executive is appointed by the legislature which is elected by the people, the legislature makes more general decisions than the executive, and the executive can be replaced upon a vote of no confidence. However, in a parliamentary democracy, the people can't recall members of parliament, and the feedback upon the parliament by the people is too weak (on the order of a few bits every election period).
Another way of separating concerns would be to do so geographically. The local government elects/appoints that which manages local concerns. Invariably some decisions will not be purely local, so these escape the system and are handled by a regional group, held accountable to the people of the region as a whole. Some decisions escape *that* system and are handled by a national group, and so on. To draw a similarity to the body: we usually don't notice what's going on with autonomous functions unless something needs our attention (because the conscious mind would be inundated otherwise); and similarly, the national group or people as a whole wouldn't have to deal with all that variety unless the regional ones can't manage.
Yet another would be by specialization. A broad organization could set objectives and a playing field and then admit more specialized organizations - sort of like a regulated market. In this approach, there is a trade-off by redundancy. If there are many organizations dealing with the same thing, they can't make use of the economics of scale, but if there are few, they may escape incentives based on competition.
These structures are not foolproof. If the regulator has too low variety, they will fail in some way or other. The implementers may hide specialist specific knowledge to steer the people in a certain direction, the national government may argue in favor of efficiency, market participants may redefine the rules, and so on. But neither is hierarchy. A bureaucratic hierarchy is quite bad at adapting to the world, because it is filled with many minor bosses who want to keep their power; and a brute force hierarchy, while very good at translating specific directives into action, can lose contact with the world as information travels up the pyramid.
Thus, no matter what model (or combination of models) is to be used, I also think we'll have to increase the variety of the organizations/state, so that these kinds of errors don't appear, or appear less often. To do so, one could use automation, either to replace or to augment the bureaucracy; but the computers would have to be used in the proper way to make the picture simpler rather than more complex.
I say that the system "has" to be anchored in the people. There's no law of cybernetics directly saying this; applying cybernetics to organizations and states just suggests how one is to make a regulator that takes some specifications and holds a system within them, and what is required for the state to do so. However, if the specifications were set by a Dear Leader instead of the people, especially if the state is of much higher variety, the dictator would become mad with power and "introduce instability". In other words, he would make such strange decisions that the people would not put up with it and there would be a revolution -- that, or the police would quash it and the nation would suffer. If the dictator disregards that and considers his own well being the only metric, then that's irrelevant, of course, so it's not an absolute, but it's pretty close to one.
no subject
Date: 2011-02-04 09:55 pm (UTC)no subject
Date: 2011-02-05 12:22 pm (UTC)So, if a bureaucracy is supposed to be impartial and carry out its tasks no matter the inputs, it's similar to a program; and the less discretion the bureaucrats can exercise, the closer the similarity. Like you say, programs can be exploited, and these "programs" can be, too.
The difference between the two would be that automation would more rigidly separate the tasks of specifying and implementing, and when they could deal with higher variety, would also be more powerful and so should have even stronger safeguards.
The decisions and actions of the government should be public unless there's a reason not to (usually involving privacy), not the other way around. This would include the data used to calculate if that is done. The programs' source should be publicly available, and for specific calculations, it should be possible for independents to verify the result by using the data (which is made public) and the program (which also is). The actual mechanism of balance is that the public (or the parts of the public that would understand the programs) would be more interested in verifying the results if the decision is controversial or the question is important, and that reasoning should work unless there's a massive conspiracy of programmers and media.
In some cases, it would be possible to invisibly bias the system. If the system of elections to a legislature was replaced with one where everybody can directly vote on bills or pass their power to somebody else (and move the power elsewhere at any time), then someone might hack the system to give undue power to certain people by a sort of salami slicing attack. (I'm not necessarily saying this is a good system, but it's one that would be impossible without computers, and so shows the point.)
If that is the case, one is left with "ordinary" investigation. The hackers would have to plan and probably contact the person in question, which would leave traces; or sysadmins might have noticed attacks on their computers, and so on. The fact that power was being siphoned would not be discovered until later, but that is usually the case with attacks and crimes in general today, too.
For some kinds of software, there would be little point in hacking it. Hacking a web forum or wiki that people use to coordinate initiatives would be annoying in that it would deny service, but it would easily be discovered.
-
So to sum all that up ... :)
An automated bureaucracy would be more powerful than a manual one, but they are similar to the extent bureaucrats can't exercise discretion. To the extent they can, people would still be able to specify after the change, and to the extent the bureaucracy is rigid, the rules that keep the bureaucracy balanced could be adapted to apply to the use of the programs as well. With transparency and freely available source, the integrity of the actual calculation could be independently verified if required.
no subject
Date: 2011-02-07 09:40 pm (UTC)no subject
Date: 2011-02-09 07:36 pm (UTC)Or at least I hope so :)
(Even if not, it may still be worth doing the trade to get the agility that automation can provide - while one can run programs on any computer, one can't make a copy of all the bureaucrats of a department and experiment with changes in laws.)