How to manage complexity

Never has there been a better example of how complex the world has become than this. Someone coughs in China and we're now locked down across most of the rest of the world.

Our world is more complex and interlinked than ever. And will continue to gain in complexity.

More than just the downside of being fragile, more complex systems become harder and harder to predict. Who knew that the largest global supplier of face masks is in Wuhan. Or that the largest global supplier of swabs needed for testing kits is in Lombardy. But I'm not here to write about covid, I want to explore complexity and what it means for companies.

We often deal with complexity and complicated information in the same way, despite them not being the same (more on that in a minute). We cope using regression, deduction and inference. As pattern seeking animals, these tools are vital to our survival. Our brains would be the fried egg in Reagan's anti drug campaign if we didn't. Problem is, we suck at turning these tools into accurate predictions.

A key reason is that to predict we rely on past and current information. This the turkey problem - each day it gets fed and everyday it does, believes that the human is there to feed it. Thanksgiving gives it quite the unexpected plot twist to that assumption, ironically just as its confidence in its prediction that the human is back with more food was at its highest.

Black Swan events are increasingly common. In no small part, it's because we're increasingly trying to make guesses in an ever more complex web. If we suck at prediction on a micro scale, we sure as shit suck on a macro one.

It's important though to understand that complexity isn't the same as something that's merely complicated. "Complicated systems have many moving parts, but they operate in patterned ways. It’s possible to make accurate predictions about how a complicated system will behave." Flying a plane is complicated. But it's predictable. You do it right and it works (unless you bought a 737 Max).

Complexity is a much more unpredictable beast. Previous behaviour doesn't predict future. Which messes with our little pattern spotting, deduction based brains.

What's really interesting about complexity, is that "what can start very simply, with uninspiring components can develop into something highly sophisticated and complex". The universe follows this pattern by the way. Moving from order to complexity and eventually, chaos. So far, it's just dragged us along the edge of chaos, getting ever more complex. And this drag along complexity is the reason a few cells in a pond resulted in me being here typing this to you - order of those cells developed ever more complex systems until, well, here we are.

This is very analogous to startups. One or more people, plodding around with a laptop, trying to make something happen. Decade later you're the largest company in the world. Something with very un-complex beginnings can create something amazingly complex.

But this progression to complexity, can well lead to chaos. If we think about natures pattern:

Order > Complexity > Chaos

Then I think it's not a stretch to map companies as:

Startup > Successful company > Disrupted/Bust

Ironically, the primary goal of most startups, to become large and dominate as fast as possible, provides fertile breeding ground for complexity to run rife and potentially, tip into chaos.

Our response to complexity just makes things worse. The standard answer to the question of "how do we manage complexity" is that we create systems. To make the complex, reliable. The irony here is that you come to rely on using approaches that are the exact opposite of what got you there. Of course, you can't not standardise some things, otherwise you will ensure chaos. But I think that most companies find the warm embrace of reliability and standardisation too much of a crutch for their need to feel in control and decide to go hard on repeatable processes over dynamic innovation.

The failure to manage complexity ultimately is a failure of leadership. This comes in a few forms.

Hired guns are often brought in as companies scale. People who've been there and done it are roped in to help the company navigate a phase of growth - read, a phase of increasing complexity. There are two problems with this.

First, as we've discussed, a previous success is not a reliable indicator of future success. Second, most leaders are overconfident in their ability and arriving shooting from the hip in an urgent quest to prove their value to their new overlords. "Most executives believe they can take in and make sense of more information than research suggests they actually can. As a result, they often act prematurely, making major decisions without fully comprehending the likely consequences for the system." A failure to arrive as someone who is there to learn, rather than to teach, can have a dramatic series of unintended consequences. The teacher quickly puts in place systems that do more harm than good as they’ve not taken the time to really learn before they act.

There's another form of failure in leadership - the disenchanted innovator. Entrepreneurs are by definition restless souls. The skills needed in the early days, to be scrappy and in the trenches are markedly different to the steady hand needed to take a company forward at scale. This can manifest in a few ways. One is just that they cannot learn to develop their operating system to accommodate the new requirements. The other is that they don't want to. Even if they can make this jump, there can be a real disinterest in being that new operator of a more sophisticated and slower moving ship. I think there are a number of founders who long for the days where things were simpler, sad in some sense that they're to blame for the gilded cage they now inhabit.

Of course this is solvable. A new CEO, a new venture, a quick exit - but for many, the level of self acceptance and outright balls needed to admit to themselves that they've created a cage and not the freedom they desired is beyond them - their concept of themselves as a person now far too intertwined with that of their role at the company.

So what can we do? What options are available to stop us choking on complexity? Or strangling innovation by managing it with rigidity?

There are alternative approaches to managing companies. Holacracy being the most famous of them right now. Zappos was a major innovator here, removing all management to adopt the circle based approach. However it looks like a case of buyer beware - it's not been a resounding success. Zappos too have adjusted, reintroducing managers in some areas to manage the self-managed organisation. Medium, who also flipped to holacracy reversed the decision, citing how difficult it is to actually make work.

A wholesale shift away from traditional organisation structures smells like a complex answer to how to stop complexity stifling the company. What if the solution was altogether simpler?

Perhaps in fact, the answer is entirely simple - we need better leaders.

I'm tempted to end the essay there in a nod to the Sopranos and just fade to black, but let me explain a bit more.

It is striking that the desire to control comes from a need to feel in control. We can see this desire to control outcomes across our lives [1]. But perhaps the answer is to better mediate our need for control with more autonomy.

Coming back to our friend nature, we see that too much top down pressure stops progress. Too much bottom up ends up in chaos. We have to find a balance. One where leaders focus on understanding. One where they can set directions, set narratives, set stories. "The basis for sharing and building on this understanding of complexity is through stories and bottom-up working models, not top-down, hierarchical, constructed systems that everyone has to ‘buy into."

Better leaders mean those that are comfortable with not knowing, not having all the answers. Ones who truly believe that their team has the answers and are willing to release control to empower them to put these into play. That ability to empower people to make change, even if they are the most incremental of progressions has a massive delta when it comes to a happy, creative, motivated and engaged workforce.

I used to love watching a series called Undercover Boss. They'd take a CEO of a large organisation, disguise them and give them some menial low level job. Without fail, these CEO's had multiple come to Jesus moments. Realisations of how processes and procedures implemented by working groups were getting in the way of exactly that - productive work. The show would end with the big reveal and giving pay rises and cars to the employees on the ground they'd worked with. All a bit crap, but the point remains - the answers are often at the bottom, not at the top.

Our answer to how to evolve, how to grow, how to navigate the complexity as we scale our organisations doesn't come from us finding the answers. It lies in our ability to empower others to create the answers.

Perhaps then the antidote to the negative forces of complexity is simple to write, harder to implement. We need curious leaders who are humble in their capabilities. We need to find purpose that goes beyond profit and develop shared stories that can thrive, supported by and not stifled by hierarchy. And a comfort that we can't control everything.

This seemingly intangible solution starts quite tangibly - with you. We hire people and resource teams using our own lens of what good looks like. Your need to control will dictate how controlling your leaders are. Your need for the answers will dictate how pervasive the processes become. Your overestimation of your capability will mean you miss what's right in front of you.

If we want better leaders to better manage the complexity that arises, the best place to start, is with you.

[1] The rise of practices like mindfulness and other eastern philosophy’s feels directly linked to our need to feel in control in a world that often feels out of control

Get more content like this. Every Sunday.
Join the newsletter below.
← View all posts