How Complex Systems Fail

In this essay I propose to show that three complex systems all fail by the same process. I propose that this process takes six steps:

1. Genesis. The system begins with a process that is able to extract resources to sustain itself.

2. Simple Expansion. The system uses its capabilities to expand and consume all readily accessible resources.

3. Specialized Expansion. The system develops specialized variations that permit it to consume additional resources.

4. Maximum Specialization. The system attains maximum specialization permitting optimal consumption of resources.

5. Perturbation. The system suffers a perturbation in its environment that disturbs its operation.

6. Collapse. The system is unable to cope with the perturbation and collapses on itself.

Example 1: Ecosystems
Let us consider an isolated ecosystem. Imagine, for example, a newly-created volcanic island. At first it is inhospitable to life, but with the passage of time the rocks degrade into soil sufficient to support hardy vegetation. Over thousands to millions of years, creatures arrive at the island by flying, rafting, or floating. Many fail to take root, but some prosper. Initially, plants are the only inhabitants of the island, for they are the only creatures capable of extracting negentropy from the available resources: minerals, sunlight, and water.

After a short while, there are several different species of flora on the island. These must compete with each other for the three basic resources. For example, some plants will develop stronger stalks to enable them to grow taller, hogging more sunlight for themselves. Other plants might develop deeper root systems to better obtain minerals and water. This competition will generate greater specialization among the differing plant species.

After the island builds enough foliage, some animals start to move in to feed on the plants. The animals enjoy great initial success, for they have no competition for the food supply. Some of the plants will develop defenses against the herbivores, such as developing poisons or thorns: more specialization

Eventually other animals arrive and now must compete with each other to harvest as much of the negentropy in the flora as possible. This will entail even more specialization, as some animals develop resistance to the poisons that some plants use. Some herbivores will develop bigger and stronger teeth to mash up more vegetation. Some will develop schemes for reaching higher plant life, such as longer legs or trunks for reaching higher into the foliage.

Next will come the carnivores. These animals will feed on the herbivores, forcing more specialization among the herbivores to avoid being eaten by the carnivores. The carnivores, in turn, will be forced to specialize to harvest particular species of herbivores. 

Eventually the ecosystem will stabilize at an optimal level of specialization that permits maximum utilization of the available resources. Every microenvironment on the island has its own microbiome that balances all the competing interests of the different species to insure that the whole thing is in perfect balance.

Then one day something changes. Perhaps a slight shift in climate changes the flow of air over the island, and it becomes a bit drier. Not much, mind you, but still enough to shift the balance. If it’s a tiny change, the ecosystem can adjust with a few changes in the ratios of different species. Some will die back a bit and some will expand a bit.

But what if the change is bigger? If the perturbation is large enough, it might well drive one of the species to extinction. If that happens, then the species that depended upon the now-extinct species themselves face a crisis; if they were well-diversified, they’d simply shift to reliance on some other species. But that would not be the case in a highly specialized ecosystem. A truly optimal ecosystem will have each component specialized to perfection, completely dependent upon the conditions that existed—until now. Thus, a perfectly optimized, highly specialized ecosystem will be like a line of dominoes: if one component of the system fails, then some species go extinct, driving other species to extinction, then other species, and so on until the ecosystem has completely collapsed. 

A specific example of this concept can be found in my essay from 1998 entitled “Koalas Versus Goats”. 

Example 2: Financial Systems
Capitalism is a system for pooling capital and directing it towards the most productive activities. Its genesis was in the early trading voyages for spice in the Far East. These voyages were inherantly risky, but immensely profitable when successful. If you sank all your money into one ship and it sank, you lost everything. Nobody was rich enough to fund half a dozen ships, but a bunch of people could all get together, pool their money, and as a group they could fund half a dozen ships. Some would sink, but some would make it back, and you’d make a ton of money. Of course, this required legal documentation to insure that everybody’s contribution to the enterprise was properly recorded so that when the ship came in a few years later, you’d be able to collect your profits. It also required the development of a proper legal system, with all the necessary laws, lawyers, and courts to adjudicate the inevitable disputes. But the concept of “stock” arose from these arrangements. You purchased so many “shares” of the total cost of the effort, and received a portion of the profits in proportion to your shares. 

Financial systems are the mechanism of capitalism; they are merely schemes for arranging the flow of capital from investors to companies through a vast network of pipes, valves, storage tanks, and pumps. As the world has grown richer, the supply of capital has grown and the plumbing has grown with it. In pursuit of profit, ever more complex plumbing systems have been established. If you have money to invest, but lack the time to research the various business opportunities, you can give your money to a mutual fund that does the research and invests for you, taking a fee for its service. 

It gets a lot more complicated from there. The financial system has developed ever more complex systems for channeling money. The purpose of these complications is to wring slightly greater profits out of the system. Competition is intense, and the slightest edge can produce massive profits; hence the proliferation of complexity.

The central weakness of financial systems is their dependence upon confidence. Confidence creates money. Here’s a simple example of how confidence works. When the economy is booming, financial institutions borrow more money so that they can make more investments. But if the economy suffers a downturn, then those same financial institutions discover that their investments are worthless and they don’t have the money to repay the money they borrowed. The institutions they borrowed from can’t get their money back, so THEY can’t repay their own debts, and the failure spreads through the entire financial system. This happened in 1929; financial institutions collapsed in a domino effect, and the consequences spread through the economy until total GDP dropped by 30%. Something similar almost happened in 2009, but the government stepped in and bailed out the financial sector with huge expenditures. With the Covid-19 pandemic, the global economy is facing the most severe financial crisis in history, and the odds of a huge depression are high. If our financial system were much simpler, it would be able to weather this storm. Of course, if the financial system were that simple, we wouldn’t be able to finance the economic growth that we’ve been seeing for the last fifty years.

Example 3: Civilizations
We have many examples of complex systems collapsing: civilizations. There have been many such civilizational collapses:

1. Harappan
You probably have never heard of the Harappan civilization. It flourished for two thousand years during the Bronze Age along the banks of the Indus River in what is now Pakistan. It arose at roughly the same time as the big civilizations in Mesopotamia and Egypt. Unlike these civilizations, however, Harappan civilization collapsed about 3,000 years ago. Why? If there are N scholars studying Harappan collapse, then there are N + 1 theories as to why it collapsed. 

2. Mayan
You probably DO know about the collapse of Mayan civilization. This was a flourishing civilization in southern Mexico that suddenly went “Poof!” and was gone.

3. Anasazi.
A civilization in New Mexico and Arizona. Again, it just disappeared.

4. Rome
Well, DOH! The fall of Rome is the classic {pun intended} example of civilizational collapse. Rome was the mightiest empire in Western history—until it wasn’t. Again, there are a zillion theories as to why it fell. I’m told that college courses in ancient history often challenge students in the final exam to concoct their own theory as to why Rome fell. 

I’m currently reading a book entitled “The Collapse of Complex Societies”, which presents the many theories of collapse. There are so many that the author even creates a system of categories of theories to describe them all. So far, however, the author has not addressed the point I make here: that it is complexity itself that destroys civilizations. Yes, yes, yes, I know: it was the barbarian invasions that destroyed Rome. Or the institution of slavery. Or Christianity. Or poisoning from lead pipes. I maintain that these were merely the perturbations that triggered the final collapse; the final cause, in my opinion, was the fact that the society had developed to such an optimum of specialization that it couldn’t cope with a perturbation that it would have shrugged off in its youth. 

For example, the Celts actually captured and sacked Rome in 387 BCE, but that invasion didn’t destroy Rome; the city bounced back and went on to wipe out many of the Celtic nations. But when the Ostrogoths sacked Rome in 410 CE, Rome was already on the way down. 

The difference between Rome in 387 BCE and Rome in 410 CE was the early Rome was still a rough-and-ready place with an ‘anything goes’ society, whereas Rome in 410 CE suffered from severe social hardening of the arteries. An immense body of laws had accumulated over the centuries, laws regulating almost every aspect of life, and many of them inconsistent (in 530 CE, the Emperor Justinian of Constantinople had the entire collection of laws recompiled and rewritten into a single, orderly code). But the law was just one symptom of the social complexity that froze everything in place. Governance had become disorderly, with lots of factions vying for power. Between the Senators, the Roman mob, the wealthy, and the Praetorian Guard, there was constant chaos as power shifted from faction to faction. The army was a mess of foreign recruits speaking random languages, with little in the way of central control. The economy was caught up in a maze of taxation, bribery, and shady dealing. Putting it all together, we find that we can’t put it all together; by the fifth century, the Roman Empire was a crazy, chaotic mess financially, militarily, politically, and legally. It collapsed in its own chaos.

Conclusion
My thesis is that complex systems always evolve towards greater efficiency by increasing their specialization, but the specialization that optimizes their performance makes them increasingly vulnerable to perturbations. Eventually a perturbation comes along that the system can’t handle, and once things start to unravel, the system must collapse all the way back to its virgin state before it can recover.