The Engineering Manager

The difficult second album

Question: what do Nirvana, Led Zeppelin and the Beastie Boys all have in common? Answer: they had brilliant sophomore albums. Nirvana released Nevermind. Led Zeppelin dropped Led Zeppelin II. The Beastie Boys delivered Paul’s Boutique. These are classic albums that have stood the test of time. They were much, much better than their first albums. They were artistically accomplished and refined.

The sophomore slump, as it is often called, is a make or break situation for new artists. A first album may have been years, if not a whole life, in the making. The second album must be made with the burden of a record deal that may contractually require a new record being made every 2 years. The pressure is on to deliver something as good, if not better, in a shorter timeframe with higher expectations. Does this sound familiar at all? If the second album is a knockout then that’s proof of a successful artist. Success begins to grow exponentially. If it’s mediocre, well, let’s just not talk about the Second Coming by The Stone Roses.

If your company has been a success due to a stellar first product and growth is beginning to slow, then people will begin to ponder whether it’s time for a second product. Can the company have the same success all over again? The notion can generate a great deal of excitement for the business: salespeople will have something new to sell to a potentially untapped market, and some of your engineers may finally get that greenfield project that they’ve been waiting for.

However, the reality is that launching the second product is extremely hard.

• High expectation of success: There is probably an implicit expectation that anything new that gets created will be just as successful as your first product. After all, it’s the same successful people making it, right?
• Underestimation of effort: Your existing product has probably gotten to a stage where it has had enough time, thought and planning to have a clear roadmap that is releasing at regular increments, and all of the under-the-bonnet infrastructure, tooling and frameworks are already there. Starting from scratch gives you none of these force multipliers: it may actually take longer than expected to get something off the ground.
• Umpteen existing processes: Your engineering and company processes have been created around your existing product. How is this going to work when there is a new product being made, potentially addressing a different market, with a different release cycle and different revenue model?

Doing something new is risky and it might fail. Think about how many startups fail every year. Your second product might suffer the same fate. How do you proceed in such a way that allows the best chance possible for your new product to survive?

Let’s consult the literature.

The Innovator’s Dilemma

The good news is that this is not a new problem. It was written about extensively way back in 1997 in a book called The Innovator’s Dilemma. This article isn’t meant to serve as an in-depth book review, but it is easy to distill the principles. I would highly recommend reading it regardless; the book is as relevant now as it was when it was written.

It addresses how successful companies can ultimately fail in the face of disruptive innovation. Imagine this: given you have a successful product which is selling well and iterating through a sensible roadmap with positive feedback from your customers, how can it be the case that a small competitor’s offering, which is seemingly worse-looking, slower and with more bugs, is beginning to eat away at more of your deals?

The answer is that the smaller, disruptive company is better focussed on satisfying the future needs of your customers rather than their current needs, which is what your own product is serving. An idea for a niche market may be dismissed because it doesn’t appear to be worth the time and money when compared to further developing a current product. This thinking, left unchecked, can be similar to the market shifting from Blockbuster to Netflix. Given a few years to simmer, this can turn into a world of pain.

This situation that is described so well in The Innovator’s Dilemma is exactly the reason that new products have difficulty growing from companies that have an existing successful offering. Two things can happen:

• The no-starter: After identifying a cool new product idea that will serve a new market, it is dismissed because the current product is already making a healthy revenue and the decision is that resource is better utilized on that instead. Why would the company take a whole team and risk building something that might fail when everything is currently going well?
• The premature failure: A new product is developed and launched, but at the end of the year, when the revenue is compared to that of the main product, it is seen as too small of a percentage to be worth the continued effort. It is sunsetted and the team returns to working on the main product.

Both of these situations are inherently bad. For every excellent idea that is not started, or not given enough time, there is a gap in the market for a competitor to break through. One of these ideas, with time, could be bigger than your entire company.

Giving your second product room

How can a new product be given the space to potentially succeed?

Let’s think about a start-up. A small group of people, with full creative control and a passion for their idea, usually raise some money. With that money they’ll either:

• Succeed by breaking even and take their financial destiny into their own hands.
• Run out of money but have growth that attracts further investment.
• Run out of money and quit if it’s not going to work.

When contemplating a new product, you can untangle it from unfair comparison with your existing offering by treating it as if it were a separate company. This new product simply becomes a finite investment of time and money. But how do you decide whether it’s worth doing?

The pre-requisites

Here are some decisions that you and the business should consider carefully.

1. Decide on the financial investment. If your existing company was going to place a bet on a new market and a new product, how much would they be willing to part with? £250K? £1M? £5M?
2. Set a growth target. What is an acceptable growth rate for a new product? Is it 150% year on year, or 50%? What would look attractive to an investor if this really was a start-up?
3. Decide on the allotted time. With all of the above taken into consideration, how much time is the company willing to commit to making this a success? How long are they willing to wait for this product to break even? A quarter, a year, three years?
4. Build a financial model. Given the investment and amount of time allowed to break even at a certain growth rate, what does that mean for the amount of money available every month for CapEx, OpEx and salaries? A quick spreadsheet can help you derive these figures.

You can now all look at the data and make a call. Is this worth pursuing? If not, then that’s fine. You’ve analyzed it quantitatively and can back up your decision not to proceed. If it is, then get planning.

The go-ahead

If the business is happy to “invest” in this new idea, then it’s time to start thinking about resources.

1. Form a cross-functional team. Taking into account how much money the company would be willing to bet on this investment, how big should the team be? Think about the salaries of the engineers that you are picking. Is this a team of 3 or 6? What current projects may slow down or stop as a result of transferring staff on to this initiative? Who is your product owner going to be? They’ll need to be self-starting and have a good understanding of what it’s like to be a “mini CEO”.
2. Get buy in from other parts of the business. It can be extremely motivational to have a dedicated salesperson and marketer who can work directly with the team to bring their product to market. It doesn’t mean that they need to begin selling to brand new customers, but they could perhaps upsell the new product into existing deals. Leverage what you have.
3. Make them fully accountable for P&L. Here’s the powerful bit: tell the team that you don’t mind how the product is built, as long as it is built to a level of quality they are happy with. That team will report back on their profit and loss, and the rest is down to them. Full autonomy drives creativity.
4. Let them know it’s OK to fail. The business has outlined how long they’re willing to run with this initiative until the money runs out. If it doesn’t succeed after that, then that’s OK; not everything will be a success. Your staff will still have a job too! However, if it does succeed, they’ll be the owners of their new domain. They’ll have a product and revenue stream that they have created themselves, and they’ll have learned a huge amount about running a business along the way.

But why?

This may seem like overkill. A financial model? A completely autonomous team? Surely it would be easier to just define the new product and treat it like a new feature, funneling it into an existing delivery team? Well, you could do that, and it might work. But The Innovator’s Dilemma suggests that you will face a much higher chance of failure. Your existing processes, people, ways of working and choices of tooling suit your current, mature product. A smaller, hungrier product needs more nimble and self-directed ways of working, just like your company did right back at the very beginning.

Giving a new product complete autonomy aside from reporting on P&L allows staff to rethink how they work. It allows them to try new things with no risk of upsetting the status quo. It allows room for hustle and camaraderie and the ability to go through a brand new experience together. You’ll be surprised at just how fast things can happen with a small group of motivated people with autonomy. It’s exciting.

How does your own company spin out new products? How well does it work?

Keep on pedaling

Cadence is a term used to describe the rate at which a cyclist is pedaling. Different cyclists find their efficiency in pedaling at different rates. Looking at the professionals you can see that Chris Froome has an unusually high cadence whilst Nairo Quintana tends to cycle at a much lower cadence. They’re both effective riders despite this difference.

As each cyclist has a preference for the cadence that suits them most. When the gradient of the terrain on which they are cycling changes, they switch gears up or down so that they can maintain it. Getting the gearing wrong can be catastrophic as it causes lost ground and wasted energy.

If the gear is too low, then the cyclist pedals too fast – what is known as spinning – and a lot of energy is consumed for a negligible amount of overall movement. If the gear is too high then the cyclist can find themselves having to push extremely hard to move the pedals – known as mashing – and again, energy is wasted. When you mash going uphill it’s very hard to change gears because you can’t turn the pedals fast enough.

Cadence in software delivery

Similar to the way that cyclists need to discover which cadence is right for them, you as a leader need to define what your cadence of delivery should be for your team, division or department. What sort of projects are you expected to ship and at what frequency? What defines a good quarter or a bad quarter? A good year or bad year?

The right cadence for your department will depend on a multitude of factors, such as the industry that you’re in, the speed of the competition, the way that your software is delivered (e.g. SaaS versus on-premise), and the number of staff that you have. You may also find that the executives at the company have a predefined expectation of how many big launches are expected per year, for example, and how fast other features should iterate.

If you’re new to the company, it’s good to solicit opinions on what is expected. Is it one headline-maker per month, quarter, or year? Are your users expecting increments every week, month, or quarter? How do you plan with longer-term, risky, innovative initiatives that might fail?

Others expect steady cadence

Regardless of what the expectation is for the cadence of delivery coming from your department, I’ve found that there is one very important thing: those outside the department are very sensitive to your cadence changing. Life is more comfortable for everybody when there is a predictable flow of completed projects, rather than a packed quarter followed by a quiet one, followed by an average one.

Now, people aren’t silly: they understand that some features are small when compared to building a whole new product or revamping the entire back end infrastructure. However, having a steady and predictable cadence is important.

• Marketing are allowed the time to plan and execute their webinars, advertisements and events. If your output is spaced predictably, they too can predictably plan their time. Too many launches in close vicinity isn’t desirable for market impact: it lessens the individual impression of each launch. A steady cadence makes their pipeline easier to manage and the results of their activities more measurable.
• Sales rely on the your cadence for their ammunition. Walking into a pitch knowing that the Engineering department is shipping predictably and often is a huge confidence boost. When working on important RFPs, knowing that Engineering will be popping out a steady stream of features during the process can be the deal breaker when you are compared to the competition.
• Engineering themselves love a predictable cadence. Your teams know that there will be plenty of times to launch and celebrate throughout the year and to get their teeth stuck into different parts of the system.

You’ll need to ensure that your selection of ongoing and upcoming projects maintains a steady cadence. That means having smaller projects delivering while long-burners tick away, iterations of existing functionality while technical debt is fixed, and that your headline launches are timed well for the benefit of the company.

Maintaining cadence

Categorizing your projects

If you have an idea of what is expected for the cadence of your department, then the next step is to work out what an ideal flow of projects would look like at any given time. These categories of project can be balanced in such a way that you are always shipping something at regular intervals, regardless of whether it is big or small.

Have a look at the categories below. Next to each is an indicator of visibility and size.

• Big ticket products and features: These are big, shiny additions to your overall offering. They could be a whole new product or an advanced differentiating feature. These can involve new and uncertain work, and will have a very visible impact if the deadline is missed (e.g. they are to be unveiled at an event, or there is an in-depth marketing announcement planned). These are of high visibility and of large size.
• Iterations on existing functionality: These are incremental improvements to what you already have. This could include new visualizations of existing data, a slicker UX workflow for a part of the application, or a revamp of the login screen. Medium visibility, medium size.
• Bug fixes and small improvements: Fixing bugs that customers have reported and tweaking what you already have. For example, fixing the format of some data being returned in an API call, making the delete button not occasionally throw an error, fixing a situation where a pop-up can get stuck. Low visibility, small size.
• Architectural work: Migrating or redesigning storage, horizontally scaling a previously monolithic process, rewriting an API. These are the long-running initiatives that ensure your future health, but may deliver nothing more than feature parity. Minimal visibility, large size.

Getting the balance right

The exact composition of your teams and the number of important projects will change over time. However, as a leader in Engineering you will need to make sure that the balance is right. There are dangerous situations that you will want to make sure that you don’t get into.

1. Neglecting work streams, where overall focus is continually biased on some areas resulting in other important streams of work becoming neglected.
2. Not tackling technical debt proactively, which causes big problems down the line that are often much harder to explain to other parts of the company.
3. Going long and thin, wherein a lack of bold decision making – typically being unable to say no – means that you have too many concurrent projects, resulting in them being understaffed and slow.

Let’s look at these in turn.

Neglecting work streams

This is a very easy trap to fall into. Running a SaaS software company requires continual improvement of your application estate. The industry and competition moves fast and, most importantly, your users are used to incremental improvements in other software they use.

Ensure that all areas of your application are revisited for refinement with time. Even better, have enough teams so that all areas have a clear owner, and that they work to KPIs that encourage continual improvement. Ignoring them leads to rot in the form of technical debt.

Technical debt

This must continually be tackled, in all forms: upgrading old infrastructure, redesigning and rearchitecting for scale, and ensuring that code is continually refactored as new code is added. Empower your engineers to be confident enough to estimate tasks so that they can always pragmatically follow the Boy Scout rule.

Also empower your engineers to tell you which parts of the application estate are due an overhaul and why. Make plans to do so while you ship other things. Not only will this work make your application more scalable and prevent future crises, but your staff will be motivated to make a difference if they are the ones suggesting it!

Always, if possible, have technical debt being tackled in some capacity all of the time. Your future depends on it: increasing amounts of technical debt will make new projects even harder to build.

This makes it akin to mashing in our cadence metaphor: the pedals get harder to turn as the hill gets steeper, and it becomes even harder to change gear.

Going long and thin

It’s also very easy to go long and thin. As a leader you will need to practice saying no to incoming demands so that you can keep your projects moving at an acceptable pace. Whilst it can be easy in the short-term to say yes to that extra favour from a client or the CEO, it further dilutes the amount of staff that you have to work on each work stream.

Although this creates the illusion that you are being more effective and working on even more projects, the reality is that you’re just getting them all done much more slowly, becoming more exposed to people being ill or going on holiday, and setting yourself up to apologize for being late down the line.

Too many concurrent projects is akin to spinning in our cadence metaphor: lots of work for not much reward.

A good mix

So consider what a good mix is for you. This will shift depending on the life stage of your company, but regardless, the advice is actually quite simple. At any given time, balance teams producing low visibility work with others producing high visibility work. For teams tackling very large projects, ensure that they are covered by other teams shipping small projects.

When you hit it right, it will feel like your projects are flying out the door all year round. That critical year-long rewrite of your backend is protected by tens of smaller features rolling out. When you hit it wrong, you may appear to ship nothing despite putting a lot of effort in, or you may produce a lot of low impact work whilst ignoring the bigger issues at hand.

In summary

We’ve looked at the idea of cadence of delivery, and how it is important it is to not only ship, but to ship predictably often. You need to balance ongoing priorities and projects, continually tackle technical debt and also be confident in saying no if it makes you oversubscribed.

Where do you stand with your current cadence? Do you feel that you are staying steady, regular and efficient, or are you spinning or mashing? If so, why do you think that is? Was it pressure from others, or was it just down to good or bad planning?