So earlier this week, Rohit posted Meditations on Barbells which is his argument that nature tends towards bimodal distributions if things are not periodically shaken up. His argument is sound and convincing, but he ends on the note of “So we should shake things up to prevent this strong bimodal distribution”. Now I agree that shaking things up is not a bad thing, if it results in the bimodal distribution getting removed without causing other externalities to create a worse equilibrium. The problem is two fold: how does one actually meaningfully shake things up, and how does one do so without creating harmful externalities.
Sadly, in this post I’m not going to explore how to solve that problem. I’m instead going to share some of the harmful externalities I see from attempting to shake things up. The law of unintended consequences likes to work with Mr Murphy to bite folks in the butt when they trying to do radical work, and that is my concern with shaking things up.
Failure modes
Those who have spoken with me for a while know that when I discuss radical reforms, my concern is not with the potential benefits, but the failure modes of the changes. Take, for example, one of the most radical changes: a revolution. A political group rises up and tears down the current government (it doesn’t matter which one), and starts working to put their own government in place. How does one ensure this government is actually better than the current one? The failure modes on revolution are exceedingly “is replaced by some form of autocracy which oppresses the people” and the success modes are “is about the same” or (very rarely) “something better”. And this isn’t my personal opinion, this is the evidence of history. Those who want to establish a new government always say “This time it will be better” but are almost never right. (There are reasons for this, but I’m not trying to analyze the types who try to lead a revolution here)
An inverted pattern occurs with new technologies. Either it utterly fails to get out the gate (high probability), it joins the market but never really takes off (often failing after a few years to some better known alternative), or it is actually wildly disruptive and changes humanity forever (pretty rare, often with a long runway). The likelihood that any given idea is actually going to be significant is pretty small. Once you cross that, the results for how it will affect society are pretty mixed, usually with both good and bad, and ascribing any specific value to the results is often a wash.
Legitimate Crisis
A lot of people like to think that because something has been around for a while, it will be around for a while more. This is, in general, accurate. However, it is not accurate when you starting changing the thing in question. One of the key factors that keeps political institutions running is legitimacy. In this case, we’re not talking about if it’s really your kid (although that plays into the historical notions of the topic) but of political capital and the validity of a government to rule its people. If a government changes how things work, that costs legitimacy. If enough legitimacy is lost, people will riot and revolt. (which is to say that if you change things a lot, but people just go along with it, you haven’t spent all your legitimacy yet).
But Wo’ah, how does one measure legitimacy? Well, obviously it’s not a coin or other currency. It’s closer to approval polls (although these don’t fully capture legitimacy, one could have low approval (I don’t like what you’re doing…) but high legitimacy (…even so, I don’t think you should be stopped)). So, in short, it’s very hard to directly measure legitimacy, but there are obvious warning signs when its running low (riots in the streets, storming the capitol, etc). It also can be very broad (the entire national government is horrible and corrupt) or very isolated (the FDA is corrupt and untrustworthy). (The FDA might not be the best institution, but I don’t have a good more local example).
What can a crisis of legitimacy cause? Well it’ll shake things up, that’s for sure. As I indicated, it can lead to rebellion and riots, but also distrust of courts, law enforcement, laws, defection in the military, opens the door for espionage, and a host of other things. Basically it creates the belief that in spite of these institutions having existed for a while, their current behavior leads one to believe they will not last much longer. That, often, is a self fulfilling prophesy, since as people are doubtful of the legitimacy, they lean of the institution less, which weakens it, which results in less legitimacy… on and on.
Cascading Problems
Cetaris Parabis. “All things being equal.” A common philosophy where one uses a control across multiple factors to isolate the effect of a single factor. The unfortunate (or fortunate, circumstances dictate) fact of reality is that nothing is actually independent in this manner. What caused the shipping issues? Pandemic? Laws? Labor problems? Space? I’m sure all of these contributed and more. And as a result, it’s affecting dozens of (if not literally every) industries. It’s nearly impossible to change or shake up just one thing.
And that is a real risk: you intend to change one thing, and the cascade of unexpected results follows and we end up with part lead times hitherto unprecedented. Of course, that’s not the only direction it could go: one can invent the airplane and within a few years, transport across the ocean is measured in hours, not days or weeks, granting previously inconceivable freedom of movement and business opportunities. My concern with any intentional attempts to shake things up is the domino effect of changes that one change can create.
Into the Twilight Zone
This isn’t to say we shouldn’t seek ways to change things and disrupt the status quo. This is all about doing so with eyes wide open. Eliezer Yudowski talks a lot about aligned AI and how we need to be intentional about our AI development because AGI poses an X risk (which is just a fancy rat term for extinction risk). Others, like my mother, talk about the risks of global warming and how that will threaten many lives. Others still speak about China and the threats of war looming on the horizon.
Honestly, these are all valid concerns (whether I regard them all equally is another matter, they’re still valid). And I think that things should, must, change. But one should be pursuing such goals with an awareness that things do not always (almost never) go the way we plan. Fixing a flaw in government could cause a legitimacy crisis, or have unexpected and awful failure modes, or could topple the very things that keep the lights on. Doing our best to look at not only what we want to fix, but what could go wrong if we get our way, exactly the way we want it, but also the way we’re most likely to get it, is the best way to avoid having such things actually go wrong.
Sometimes, crazy works. The rest of the time, prepare for the worst
-Wo’ah the Wise
Disruption Risks
Really appreciate the commentary and thoughts! Few comments:
1. > Those who want to establish a new government always say “This time it will be better” but are almost never right. -- I disagree with this. Our institutions today, considered in any long aec of history, are better, at least eventually.
2. A microcosm of this might be corporate reorgs, which provide both positive and negative insights, though at least proves reorgs and shaking up need not be fatal or detrimental to legitimacy. Accenture is one that's done this to grart effect.
3. Most long standing institutions do evolve. They change their mandate, their charter, their methods and more. I'm just suggesting they also add their own setups to that mix. I admit there are practical problems aplenty, but I'm not convinced it's a legitimacy crisis.