How Systems Fail

I want to discuss the behavior of large systems of competitively interacting agents. Such systems exhibit unique properties that can lead to surprising (and often tragic) results. Two ideal examples of such systems of interacting agents are the stock market and an ecosystem. Less appropriate examples include a polity, a corporation, a biological system such as a human body, a power grid, and a computer program.

Next I shall define some special terms for use in this discussion:

Competitively interacting agent: a distinct entity, endowed with some purpose or goal and adjustable behavioral means of achieving that goal.

Objective contextual factors: Factors affecting the system that are outside the direct control of any of the interacting agents in the system. In an ecosystem, for example, objective factors might include the average temperature and rainfall. In a stock market, objective factors might include the inflation rate and interest rates.

Subjective contextual factors: Factors affecting the system that are within the direct control of some of the interacting agents in the system. For example, in an ecosystem, subjective factors might include the coyote's preferences for rabbits over cats, or rabbits' olfactory sensitivity to coyotes. In the stock market, subjective factors might include the price of any particular stock.

Context: The current state of the system. This encompasses both objective factors and subjective factors.

Perturbation: A change in the context of the system. Perturbations come in different sizes; the size of the perturbation determines the degree of stress that it places on the system.

History: The past record of performance of the system, skewed towards the present, and weighted by the effective memory of the interacting agents in the system. Thus, stock market participants will pay close attention to the history of the stock market over the last few years, but pay less attention to the history of the stock market many decades ago. An ecosystem has a much longer memory. Especially significant is the history of the context of the system.

Confidence: The degree to which the history of the system is perceived to demonstrate small perturbations and a stable context.

Now let's assemble these customized terms into an analysis of the career of a system. We'll start with a baby system, just emerging from whatever primordial mists constitute its prehistory. Following an initial anarchic/infantile period, the system will eventually reach a point of approximate stability. In other words, its context will become stable and perturbations will be small.

Note, however, that this first-round stabilization will be based on objective contextual factors, not subjective factors. That is, the individual interacting agents will initially optimize their behaviors to the objective factors. This is unavoidable because the behaviors of other agents (which constitute the subjective factors) are still very much in flux, and are therefore unreliable. Plants in a new ecosystem will hone their behavior to the amount of sunlight, the temperature, the soil, and the water supply; they will ignore the depredations of herbivores, molds, and competing plants. Buyers in a virgin stock market will base their purchase decisions solely on the dividend history of the individual stocks.

Once the system has completed this first phase and established its initial context, a new phase begins in which subjective factors begin to affect the behavior of the interacting agents. The agents start looking around themselves and taking into account the behaviors of other agents. Rabbits start worrying more about how to evade coyotes; plants develop poisons to discourage herbivores; buyers start to take note of what other buyers are up to.

Here we come to the first big idea, which I will take several paragraphs to explain: subjective factors include an unavoidably arbitrary element which in turn injects a fundamental instability into the system.

The best way to see this is to compare two games, one based on objective factors and the other based on subjective factors. The subjective-factors game is simply Rock-Scissors-Paper. You and I each begin the game with one rock, one scissors, and one paper. We simultaneously play one of our three items, and compare them. A rock beats a scissors, a scissors beats a paper, and a paper beats a rock. Because our decisions are based on strictly subjective factors, the game is arbitrary; there is no reliable strategy that either of us can pursue.

Now let's consider an objective-factors version of Rock-Scissors-Paper. It requires just one small change in the game: we reverse the relationship between rock and paper, so that rock beats paper. With this one small change, we have completely changed the nature of the game. Success in the game depends solely on objective factors; anticipation of the other player's intentions is unnecessary.

Neither game is entertaining, but that is beside the point. The point is that the objective-factors game yields absolutely certain results (stalemate) and the subjective-factors game yields utterly arbitrary results.

Of course, in any real-world system, the decisions facing the interacting agents include a mixture of objective and subjective factors. If this mixture is weighted heavily toward objective factors, then the system will be objectively stable. But if the system is weighted heavily toward subjective factors, then it will be "arbitrarily stable". The stability is founded upon a complex of cooperating behaviors among many interacting agents, a complex which is fundamentally arbitrary and thus subject to rapid change.

The stability of a subjectively-weighted system is more fragile than the stability of an objectively-weighted system because the behaviors of the interacting agents can change on a much faster time scale than the objective factors can change. It takes General Motors years to develop a new line of cars, but investors can change their minds and dump its stock in minutes. Ice ages take tens of thousands of years to develop, but it only took the coyotes a few weeks to figure out that my ducks are much easier prey than the local jackrabbits.

Now for the real rub: all such systems MUST evolve in the direction of greater subjectivity. To see this, let's compare two interacting agents with different strategies. One agent is pessimistic, the other is optimistic. To put it another way, the pessimistic agent has low confidence in the system, and the optimistic agent has high confidence in the system. The pessimistic agent behaves as if perturbations are likely, while the optimistic agent behaves as if perturbations are unlikely; it's just the ant versus the grasshopper. Aesop's fable only hints at the key point top this discussion: in the short term, the grasshopper outperforms the ant. Ants who spend their time preparing for winter don't have the resources to reproduce or make profit. The ant may indeed have the advantage when winter comes -- if he survives that long. The very act of providing for long-term defense against perturbations reduces short-term competitiveness.

Thus, in an ecosystem, the competitive advantage goes to the species that is willing to precisely tune itself to the specific context of the ecosystem. The Swiss-knife species capable of handling the occasional ice age, drought, or faster predator just won't hold its own against the lightweight competitor optimized for the conditions of the moment. The pessimists will be driven into smaller niches, and the optimists will rule the world -- until the next big perturbation.

The same thing goes with markets. The pessimists who stood on the sidelines clucking their tongues as the stockmarket soared are being reduced to financial irrelevance by the tidal wave of profit going to the optimists. A larger percentage of the total wealth of our economy is now owned by the optimists. These people will continue to grow richer and richer -- until the next big perturbation comes along.

Thus, such systems reward optimism and thereby become ever more subjective. But the secular increase in subjectivity of systems cannot go on forever. The more subjectively-weighted a system becomes, the more finely-tuned its agents' behaviors are, and the more vulnerable they become to small perturbations. To use a tired metaphor, it's like a house of cards: the higher you build it, the more easily it will collapse under the tiniest breath of air. My big point in this essay is that all such houses of cards unavoidably grow higher. Guess where that leads us?

It leads us to catastrophic system collapse. We're all familiar with the mechanisms of financial collapse: a hailstorm (small perturbation) in Tasmania destroys the wheat crop; Australian wheat futures skyrocket; several large corporate investors, which had invested heavily in Tasmanian wheat options, find themselves in a crunch; their stocks fall; the Australian stock market takes a dive, dragging down a number of American corporate investors with it, which leads to a run on their stocks on Wall Street -- you know the scenario. We all expect that at some point the problem reaches a financial firewall, a mechanism that prevents the loss of confidence from escalating. But this is wishful thinking. Once a perturbation has demonstrably altered the context of the system, confidence (as I defined it above) must decrease, which must lead to changes in behavior, which leads to diminished confidence -- it's a vicious circle.

We see exactly the same thing with ecosystems; their systemwide collapses are called "mass extinctions". Here I will deviate from orthodoxy in asserting that mass extinctions are caused only proximately by whatever asteroid, ice age, volcano, or other disaster struck. That is, the initial perturbation did not directly kill off all the critters. The Cretaceous ecosystem had been stable for 150 million years; during that time, the ecosystem had woven a tightly interconnected web of dependencies. Our global ecosystem today, a relative youngster at 60 million years, has evolved some fascinating interdependencies. Some parasites hatch and mature in one species, then feed on another species. Symbiotic relationships exist between bees and flowers, ants and aphids, and a whole variety of other species. Note, however, that the most developed synergies seem to appear in the older phyla. If we confine our view to the relatively young order of mammals, we see substantially fewer cases of symbiotic relationships. It takes a long time to weave that web of interconnected relationships.

This suggests to me that our own global ecosystem is still developing its subjective contextual factors. But the Cretaceous ecosystem must have been much further down the road of subjective factor weighting. I speculate that there were many more complex dependencies in the Cretaceous ecosystem. Thus, when the asteroid hit Yucatan, the web of interdependencies, previously a source of biological synergies, became a conduit for cascading disaster. The fault, I conclude, lay not in our asteroids, but in our grasshoppers.

Fortunately, there's a floor underneath the system collapse, provided by the objective factors. Even after an ecosystem collapses, there's still sunlight, soil, and water to sustain new life. Even after a stockmarket collapses, there are still consumers, and there are still resources, tools, and laborers. The basic components are all in place to start anew. The question is, just how far down will the collapse go? That, in turn, will depend not so much on the magnitude of the proximate cause of the collapse as the extent of subjectivity in the system. The higher they fly, the further they fall; more highly subjective systems will undergo deeper collapses.

This, by the way, suggests a simple test of this hypothesis: the magnitude of mass extinctions should be concomitant with the duration of the preceding period of stability. I'll make an exception in the case of the Cambrian extinction, it being the first.

Financial systems consist of layers upon layers of subjectivity. At the bottom layer of every financial system is the objective factor: the underlying drive of human desires. One level above this is the adoption of a universal symbolic replacement for those desires, often taking the form of some objectively finite commodity such as gold or cowry shells. The next level is the standardization of this commodity in the form of coinage; an ounce of silver becomes a crown or a dollar. At the next level, we cut loose from the gold standard; a piece of paper becomes a certificate that is worth a dollar.

I'm going to pause in my march to the financial stratosphere to explain something about confidence here. Remember, the only true objective reality in financial systems is the ongoing set of needs and desires of the consumer. The only thing you can be absolutely (objectively) certain of is that people will always want food, clothing, and shelter, and probably a whole lot more than that. You can be less certain that they will always value gold -- but it's still a pretty safe bet. Will they always value dollars? Well, that depends on their confidence in the issuing agent: the United States government. So long as people believe that the government will be fiscally responsible, they'll believe that a dollar bill is worth something. The big idea here is the importance of confidence in a particular organization: the government.

Let's continue our upward march by observing that an analogous confidence can be placed in the operation of *any* organization, such as a for-profit corporation. In this case, though, the manifestation of confidence is not a dollar bill but a stock certificate. These two kinds of certificates are closely related; each promises the bearer some form of lower, more objective value upon certain conditions.

Continuing upward, then, we have a panoply of fiduciary instruments layered on top of dollar certificates and stock certificates. There are options, derivatives, bonds, money market certificates, and on and on. It gets very complicated.

Suppose that I purchase one share of MegaCorp stock for $100. MegaCorp stock pays a dividend of $1 per year. Thus, my $100 investment yields a return of 1% -- pretty paltry considering that I can get 5% from a government T-bill, or 3% from a bank savings account. Why would I make such a foolish investment? Because I believe that, one year from now, somebody else will wish to purchase my stock from me for $110. This will give me a 10% return on my investment -- much better! Note, however, that my calculation is dependent upon my belief -- confidence -- that there will indeed be such a buyer one year from now. Whence springs this confidence? Clearly, my confidence arises from the history of the market. Since the market has been steadily rising for the last few years, I have confidence that it will continue to rise. I and fifty million other investors.

Suppose, however, that our confidence is shaken. That hailstorm in Tasmania rattles our confidence, and we all bolt for the exit doors of the stockmarket. Stock prices plummet, and we lose trillions of dollars in the process. Oh well, tough luck for the speculators, right?

It doesn't stop there. Remember, a highly developed system boasts an intricate webwork of dependencies. All those dependencies tend to link the different levels of the system together. If I lose confidence in the upward climb of the Dow, I can always fall back on the confidence I have in the next lower level of the pyramid. If the stock market stops rising, I won't earn that fat 10% return on my MegaCorp stock, but at least I can be sure of getting my 1% return from its annual dividend, right?

Wrong! That "intricate webwork of dependencies" includes a bunch of financial linkages to MegaCorp's stock price. Bank loans and corporate bonds are just two of these financial linkages. The value of MegaCorp's stock provides the security against which the loans and bonds are based. If the value of MegaCorp's stock falls, then people who own MegaCorp bonds will dump them -- lowering their price and making it more difficult for MegaCorp to raise new cash with new bond issues. Banks will be warier about lending money to MegaCorp, demanding higher interest rates and more security. Thus, MegaCorp won't be able to borrow money as easily as before, which means that it won't be able to weather cash flow crunches as easily. It's competitiveness will be reduced -- which means that its profitability falls. There went my dividends.

Thus, dividends, which in a theoretically virgin business environment are pretty much an objective contextual factor, are steadily "subjectivized" by the webwork of financial linkages. This "subjectivization of objective factors" extends its tentacles all the way down through the financial pyramid of confidence as the financial system matures and develops ever more complex instruments for linking together disparate segments of the economy.

Ah, but perhaps we could erect financial firewalls that isolate portions of the financial system, so that a perturbation in one sector doesn't lead to a cascading loss of confidence in all the sectors. It sounds good, but it doesn't work. There are a whole class of financial opportunists -- arbitragers, speculators, and so forth -- who make their living by snooping around the financial environment, looking for the slightest discrepancy between any one sector and any other. They then pounce on that discrepancy and transform it into a source of immense personal profit. This is most often seen with currency fluctuations, but the basic technique can be used between any two points in the economy. Derivatives, junk bonds, pork belly futures, and all the other confusing financial instruments are really just devices for bringing each tiny part in the economy into harmony with every other part. If we erect firewalls, we isolate segments of the economy and decrease overall system performance.

Thus, the only truly objective floor underpinning our economic system is the basic set of human needs and desires. All the other components -- gold, coinage, paper money, banks, stocks -- are fundamentally subjective contextual factors. A systemwide collapse will start at the top of the chain (just as ecological disasters tend to hit the top of the food chain hardest) and spread downwards, and the more mature the financial system is, the more dependency linkages there will be to spread the collapse lower and lower down the system, until we reach the floor of human desires for food, clothing, and shelter. That's a long, long ways down from where we are now.

Next essay: extending these concepts to less appropriate systems such as polities, corporations, biological systems such as a human body, power grids, and computer programs.