Expansion: the aftermath

The company did brilliantly two years ago. But now they’re saying you’re slow.

Last year you managed to open up nearly 40 headcount in Engineering and you managed to fill those roles with talented people, almost doubling the size of the department. Yet, another year on from there, others in the business are left wondering what effect that had.

Shouldn’t have you been able to dramatically ship more software with all of these extra people? Where are all of the features that you were previously unable to prioritize because you didn’t have the staff to do so? Why is it that a small competitor is seemingly moving faster than you?

If you have managed a department through a period of rapid growth, and especially if you have aspirations of doing so in the future, you will experience this scrutiny. You know that simply adding people doesn’t make everything faster, but what can you actually do about it? How can you get others in the business to understand?

Internal and external scrutiny

Let’s divide the source of the pressure into internal and external scrutiny.

• Internal scrutiny is probing from inside the business as to what has happened to your output after your department expanded: after all, even adding just tens of people, depending on their seniority, can represent a yearly investment from the business of over £1M. They’d like to know where that money is going and what it is being used for.
• External scrutiny is the perception of the business in the marketplace after it has seen you grow. Maybe there was a PR piece explaining how your company is continuing to expand after a stellar year, or maybe you raised VC capital. Those that track the marketplace and your competitors will observing your business to see proof of what that growth has delivered. No visible progress may suggest to the outside observer that there are problems. You don’t want to look weak to your market.

The unfortunate truth is that even if you are shipping often, you will always face questions about your output, both internally and externally. As a leader, you have two weapons against the dark forces of criticism for being slow.

The first is making sure that you’re building things as quickly and efficiently as you can, given your current circumstances and resources. That’s a given. The second is making sure that you are communicating transparently and often about what you’re doing. Engineers are often very good at building awesome things the right way, but often not as good at communicating what they’re achieving, hence the scrutiny.

Before we look at some techniques you can use, why is it that you might be naturally getting slower as you get bigger?

It’s not people, it’s productivity

Let’s clear something up first. Arguments around perceived speed can become ad hominem very quickly, and that’s a route that you do not want to go down. While it is true that bigger departments can mean that performance issues can go unnoticed more easily, it isn’t a small handful of individuals that are making your whole department slow. It’s about being a victim of your own success, having to deal with legacy systems, and the increased communication overhead of having ever more people. The more accurate way to phrase your perceived slowness is that the productivity per head is decreasing, and this is the problem that needs addressing.

So what makes this productivity drop? Let’s revisit those three areas.

1. Being a victim of your own success

As brilliant as it is to do well, it also creates more work. You will have more customers expecting a higher level of service. You’ll have to create or expand your support team. You will have to tighten up the security of your system. You’ll have to improve your monitoring, build up your out-of-hours rota, and so on. Additionally, you will be continually storing more and more data which requires continual expansion and upgrades of storage systems. You may need to rewrite whole parts of your application estate so that they scale. An ever-expanding array of work requires attention just to keep things running the way that they currently are.

2. Legacy systems

Being a victim of your own success also means that you’ve created a lot of code that is fast becoming legacy. It is impossible to predict future needs, so it is therefore impossible to create infinitely extensible systems. A new product direction, new set of features or complete pivot can highlight that your current codebase is going to need dirty great hacks (bad) or a complete rewrite (also bad) to survive.

Even if you are able to continue extending your current codebase, the ever-increasing size means that it takes longer for your engineers to understand where best to fit new features, longer to write tests that actually cover all end-to-end scenarios (if that’s even possible) and longer for new changes to be peer reviewed. There’s the added pressure that having a well-established paying (with money and/or data) customer base means downtime really isn’t acceptable any more, increasing the collective fear of making sweeping changes quickly.

3. Communication and process overhead

When you’re a small start-up, the entire business can make unilateral decisions at speed as they can sit in the same room. When you’re bigger, global, and have clearly defined departments, roles, and responsibilities, decisions can converge towards a glacial pace. This isn’t limited to business decisions: technical decisions can also suffer. A decision on how to overhaul the main codebase could affect hundreds of engineers. How do you make sure that enough people are consulted? This can mean more processes and meetings are required for consensus to be reached. This can get slow, but it’s unavoidable.

Without clear processes, meetings and steps for how decisions are made, life can get very messy. The Mythical Man Month famously outlines the effect of increasing numbers of staff on potential lines of communication. When n people need to communicate amongst themselves, there are n(n-1)/2 channels*. For example, a 400 person company has 79800 communication channels(!) We need meetings and processes to tame this, but not too few and not too many. Getting the balance right is hard.

So what can you do?

We’ve painted a pretty dire situation. What can you do about it?

There’s no silver bullet, but as a leader you only really have two tools at your disposal for ensuring that you’re going quickly enough: ensuring that you’re all building software in the best way that you currently can, and making sure that your progress and decisions are communicated as transparently as possible.

1. Develop software pragmatically

You’ll need to make sure that your engineers are being as pragmatic as possible when it comes to building and maintaining your application estate. As the company grows, so will the codebase, and the risk of it getting gnarly increases dramatically.

To begin with, coach a culture of refactoring as you go along. I like the boy scout metaphor, where each engineer should try and leave an area of the codebase they are working on cleaner than when they arrived. The key in this approach is the scope: just improve the area that you are working on. You wouldn’t expect the boy scouts to improve the entire forest after camping there. Refactor a clear path and move on through.

Actively encourage technical debt being tackled. Perhaps reserve 10% of the time in your sprints to working on technical debt issues, and as a leader, champion this decision. Encourage your staff to all advocate how important technical debt removal is for the long-term success of the department. Related to this: never be afraid of deleting features. Did it not ship fully? Delete it. Did interactions with that area of the product wane significantly over time? Make the hard call and get rid of it. It will only slow you down in future.

2. Have small purposeful teams

Zooming out from the code, you’ll also want to make sure that your teams are right-sized. They need to be big enough and with the right balance of skills so that each team can make progress on their project or feature with as little outside support as possible, yet small enough so that decisions happen quickly. A team that gets too big can become impossible to plan for, both in the quantity of work and in sticking to a theme that unites them.

Each team should have a clear purpose and reason to exist. Perhaps they build and maintain a particular product or feature (i.e. their success is customer-facing) or they are purely building infrastructure (i.e. their success is serving the department). Either way, teams with a clear purpose will have an intrinsic motivation to go fast because they will be self-motivated. They will require less hands-on management as a result.

3. Communicate transparently and often

As a leader in the department, you should not expect that if you are having a stellar quarter that the rest of the business is going to know about it. Part of your role is ensuring that everyone else is informed on your progress. You owe it to your staff to share the great things that they are doing.

There are a number of techniques that you can employ here:

• Broadcast. You could publish a regular newsletter, written for a non-technical audience (e.g. the commercial mailing list), describing the progress in your team, division or department. It sounds simple, but in my experience, people really engage with these communications and enjoy reading them. Especially so if they are light-hearted and entertaining.
• Invite outsiders in. Always give other departments the chance to engage. Invite them to be stakeholders in your projects and demos. They will then better understand the pace of progress and the difficulty of technical challenges.
• Be social. Hold regular catch-ups with those in your network to discuss everything that you’ve been up to. Ask their advice candidly as a peer and friend.
• Offer Q&A. Ask to drop in on existing non-Engineering meetings where required as a guest. You could even set up your own regular session with the rest of the business.

The idea here is that as the company grows, the noisier the company gets. You need to ensure that even if you are doing great work that people know that you are. No matter where you work, you will probably face accusations of being slow, so you need to make sure that the rest of the business is making that judgement based on the facts that you present rather than just assumptions about what you’re up to.

In summary

It’s not that people get slower as a company gets bigger; it’s that productivity per head decreases when those staff are part of a larger organization. As a leader, it’s important for you to make sure that your teams are developing software in the most pragmatic way possible and that their progress and achievements are transparent to the rest of the business. If you are open, accessible and just plain amicable as a leader, you’ll find that the internal scrutiny will develop into camaraderie as you are able to prove that you’re doing the best that you can with what you have.

The knock-on effect is that those who question your speed can engage with you in conversations that criticize the right things (i.e. resources, scope, prioritization) rather than the wrong things (i.e. not “working hard enough”, performance of individuals), as the right things can be collaboratively discussed and altered where necessary, without judgement.

*This is the same number of edges in a full mesh topology. Humans and computers have more similarities than you think!