202511141410 The Complexity of Simplicity

#souce #new

Absolutely fantastic keynote tech talk given by Bryan Cantrill at TalosCon 2025 on simplicity in programming.

Abstraction

The essence of software systems is the creation of abstraction. They allow us to build software systems that do sophisticated things. They are the shoulders we stand on and provide to others. Abstractions are qualitative. The good ones allow us to hide gory implementation details in order to build "castles atop them and tunnels beneath".¹ The bad ones leak, seeping implementation details instead of sealing them, yielding systems that are unwieldy and brittle.

One of the primary roles of any software or computer science engineering education is imparting the humility that anything works at all.¹

Complexity

Complexity blossoms in bad or non-existent abstractions. Fred Brooks famously called this accidental complexity in his essay No Silver Bullet. (Where are my notes from this when I read it? #thread #wip) This is differentiated from the essential complexity endemic to a particular problem. Complexity can explode. Accidental complexity in one component can become the essential complexity in something that must interact with it! Complexity doesn't merely accrue.

Complexity can be tamed from within existing systems, but it's a massively uphill battle. Most larger changes in complexity — positive or negative! — happen in new systems and new systems can be either emergent or engineered. An emergent (202104291540 Emergence) system is one that is implicitly or explicitly a reaction to accidental complexity in the systems that came before it. A deliberately engineered system is a system that takes on essential complexity in the problem itself from first principles. Furthermore, these systems will have and offer complexity ranging from simple to complex independently of which of these two ways it came into being.

| Constructed Systems - Complex & Engineered | Revolutionary Systems - Simple & Engineered |
| ------------------------------------------ | ------------------------------------------- |
|   Accreted Systems - Complex & Emergent    |    Rebellious Systems - Simple & Emergent   |

One of the big problems with these four quadrants is that three of them are not good. It's all too easy to fall into complexity despair, viewing complexity as a kind of entropy that only increases, propelling all of us to heat death. In these dark moments, we need to be reminded that things don't always get worse. In fact, sometimes things take a very large leap in the right direction for the better.

Here Cantrill talks about how Rust was one of these leaps for the better and I couldn't agree more with the anecdote and lesson. Importantly, it showed him that there are so many things that we can view as essential complexity that are actually accidental complexity and that things that fall in these categories can shift over time e.g., when a new language is developed.

Constructed Systems - Complex & Engineered

Constructed systems have high degrees of essential complexity and complicated abstractions, often to serve a larger functionality. Examples include airline booking systems, compilers, operating systems, or databases. These are big systems in scope, people, time, etc. — it's software-in-the-large. Complexity is a killer for these systems and they are often impacted by second-system syndrome, bloat, and scope creep.

The bad news is that our most essential and most foundational software necessitate constructed systems.¹

All systems were Constructed fin the beginning. If you wanted a database in 1990, you had to write it or buy it from someone who wrote it. The software engineering thinking and canon in the 90s is all about managing these risks. Brian Kernighan states, "Controlling complexity is the essence of computer programming." Broadly speaking, without simplicity in abstractions, Constructed systems struggle to know how to say what they won't do. For example, Kubernetes has historically struggled to know what to say no to. Focus is saying no.

When simplicity in the abstraction is a non-goal, you don't know what to say no to.¹

Rebellious Systems - Simple & Emergent

Systems that come out of engineers asking "does it have to be so complicated?" These systems overthrow what came before often by discarding unnecessary constraints. They are reactionary technology. They pride themselves on what they are not more than what they are. Examples include Unix (vs. Multics), RISC (vs. CISC), Rails (vs. J2EE), NoSQL (vs. SQL), etc. These are typically successful when the Constructed systems have so much complexity that they are mired in it and don't even know that they are drowning in accidental complexity.

Rebellious systems are risky though. We see the survivors of rebellions, not the mountain of rebellions that failed. They fail when they discard not just accidental complexity but also essential complexity. This can result in more complexity (microservices vs. monoliths, unikernels) — and worse, complexity people don't talk about for fear of perceived loyalty to the legacy system they are rebelling against.

Accreted Systems - Complex & Emergent

These are systems that no one would design from first principles. They are compromises with time, space, cost, etc. Often accreted systems served a different purpose in the past, were developed elsewhere and moved, or were prototypical and lived too long. It's possible for a newly created system to be accreted though. These are widely known to be problematic from the get go, but it's how things happened. Most things start in the other quadrants and land here after a shorter or longer time. The main challenge with these systems is that they become constraints on the systems around them. These are the systems whose accidental complexity becomes the essential complexity of those that use them. These are tech debt. Morale is low on these projects.

Bug-infested death marches to nowhere.¹

Importantly, accreted systems work well enough and it's not simple to pull them out because they do in fact provide some vital function. Often trying to replace an accreted system is so challenging that you end up with another system that's born accreted in an attempt to simplify the problem, but run into the same or similar reasons that the first one failed.

Revolutionary Systems - Simple & Engineered

There are systems that are deliberately engineered to innovate with respect to the abstraction itself. These systems solve a problem that includes the abstraction itself. A revolutionary system is revolutionary exactly because they tackle essential complexity in a way that allows others to abstract it away. Therefore, developing revolutionary systems often requires taking on extraordinary amounts of essential complexity.

Revolutionary systems are often but not always developed by small, focused teams working over an extended period of time. Their teams' contributions are so valuable because they have reverence for the past and learn what was good, necessary, bad, or unnecessary. Their roots are often in their experiences with accreted systems. Even more potently, from experiences being totally mired and drowning in complexity.

Examples: CDC 6600, System R, TCP/IP, Macintosh, AS/400, PostScript, NeXSTEP, Python, NFS, TeamWare/BitKeeper, Plan 9, HTTP, Java, Disco/VMWare, SQLite, LLVM, x86-64, iPhone, ZFS, DTrace, S3, EC2, TypeScript, Docker (may be accreted now), Rust, Oxide.

See

There are risks of revolutionary systems too though. Typically they don't know when to ship because they're so new and revolutionary. They almost always take longer than expected. Adoption is very slow because adapting to new abstractions is challenging — including changing people's thinking! There are also plenty of would-be revolutionary systems that miss the mark by providing an abstraction that no one wants, needs, doesn't solve a problem, or solves a problem no one has. One of the biggest challenges for revolutionary systems is to stay funded long enough to effect the revolution it envisions for itself.

It is very complicated to make things simple!

And I would observe that it's very simple to make things complicated.¹

Slides