Ownership

Leave a comment
Growth
Photo by Agê Barros on Unsplash.

Tick tock

How much time is wasted in the cracks between phases of work?

A feature is code complete, but the pull request is waiting on review. 

Stalled

The feature branch is deployed, but it needs QA. 

Stalled

Everything is finished, but it needs deployed to live. 

Stalled.

As a phase winds down before it becomes the next one, either due to needing internal input from the same team, or due to needing external input from another team, the person doing the work has to stop and wait.

That individual may naturally turn their attention to another task so that they can remain productive, such as starting on the next ticket.

However, an individual’s focus on individual productivity can often cause the total productivity of the group to suffer.


“Oh yeah, that’s just waiting for QA. It’s been like that for a while now.”

“Did you let them know it was ready?”

“Nah, the ticket’s been moved into the right column, so they’ll pick it up.”

Well, maybe they’ll notice that it’s there, and maybe they won’t, because they’re busy with something else.

Even highly productive individuals can demonstrate a lack of ownership over the overall goal of shipping software. Instead the individual is only focussing on their part of the task. The other parts are on the other side of the fence, and that’s somebody else’s problem.


“That one’s waiting on the other team to push their changes upstream.”

“OK, did they know that you’re waiting for them?”

“They looked pretty busy, so I thought I’d let them get on with their stuff for now.”

Sometimes work can take hours, or even days, longer than it needs to because an individual only pushes forward the part that they can control. A particular process doesn’t guarantee that complex work being done by lots of people will be completed. The people themselves make it happen by doing whatever is required to see the whole task through.

When individuals embrace total ownership of their collective output, things get done faster. This is predominantly a change in mindset, not a change in process. 

But before we explore that mindset more, what if people don’t understand their responsibilities in the first place?

RACIs

Often when roles aren’t clear, the natural thing to do is to collectively spend time defining them. A tool that many find useful is the Responsibility Assignment Matrix, commonly referred to as a RACI

Within teams, departments, or even whole companies, you can draw up a RACI to better understand the different roles that individuals take for particular tasks.

These roles are, for a given task:

  • Responsible: Those that do the work to complete the task.
  • Accountable: The person ultimately responsible for delivery of the task. They approve the work that the responsible person does.
  • Consulted: Those that have their opinion sought through two-way communication.
  • Informed: Those that are updated on progress via one-way communication.

Hopefully you can see where the acronym comes from now.

Here’s an example of what part of a RACI might look like. The uppercase letters in each box correspond to the role definitions above.

CTOVP EngineeringTeam leadDevelopersQAScrum Master
Develop softwareIIARCI
Remove blockersIIRRRA
Ensure quality of softwareCRRAI

If a team is finding that tasks keep dropping through the cracks, then taking that team through a RACI exercise can often uncover all sorts of incorrect perceptions about how things get done. Give it a go and see what happens. I guarantee you’ll have some interesting debates.

Yet, remember what we said before: people make things happen, not processes. The RACI will not solve your ownership problems, but it will stimulate interesting discussion.

People solve ownership problems by realizing that they need to take total ownership of their collective output. You may have the most detailed RACI filled in, but without a mindset of ownership, little will change.

Hiding in definitions and process

Processes and definitions create places to hide within them. If you know you’re responsible for X, why should you care about Y? After all, the RACI says so, right?

“Oh, that’s her remit.”

“Nothing to do with me.

“Yeah, the other team should probably look at that.”

“Not my responsibility. I’m busy enough as it is.”

We could imagine a situation where a RACI could be abused to encourage a “not my problem” mindset.

After all, if the QA is ultimately accountable for ensuring the quality of the software being released, should the developers care less about it? 

Of course not.


When technology companies are start-ups, there is no choice for staff but to care about everything, because there simply is no other way. Start-ups embrace total ownership through lack of resources.

But success breeds growth, causing expansion of people and divisions of responsibilities and roles. The more you divide and spread that ownership, the more gaps and barriers that form between those subdivisions. Individual remits become narrower and deeper. 

You work in your box, and they work in theirs.

Instead of the founder selling, building and fixing, you now have developers on specific teams, numerous specialties of salespeople for diverse regions and products, and multiple roles for keeping customers happy and engaged. 

The risk of this scale is the collapse of total ownership.

When the remit focussed on is a tiny part of the whole, the individual optimizes for individual performance. This can easily be at conflict with the overall goal of being a successful company. 

Your lead generation machine could hit target, but those leads could be junk. Your engineers could ship to a deadline, but the quality could be awful. Your marketing event could have record attendance, but none of the attendees convert to sales. 

Overly local focus creates local maxima, and people across the company could be kings and queens of the minor mountains without conquering anything of worth.

And, of course, fingers will point: sales can’t convert those leads because they’re junk. They think it’s the fault of marketing. But marketing thinks that better salespeople could convert them. Engineering get accused of shipping a bad product, but they blame the deadline that was set by others. 

When you only care about your little patch, you’ll dump the rubbish over the fence to keep it looking pristine.

Owning it all

Instead of hiding behind processes, or driving relentlessly towards individual KPIs, truly excellent staff take ownership of the entire thing

It wasn’t your fault, it was mine. 

Leaders take ownership for their teams, and for their departments, even if it hurts and it bruises their ego when things go wrong that were out of their control.

Ownership as a mindset spreads. It builds respect. It causes different questions to be asked.

“Oh yeah, that’s waiting for QA, but I totally forgot to ask them. I’ll do that now and pair with them on it.”

Much better.

“This is waiting for the other team to push their changes upstream. I’ll ask if they can do that now as it’s blocking us.”

Excellent.

“That pull request is still waiting for review. I’ll go ask a couple of people if they can review it now.”

Perfect.

What does beta really mean?

comments 2
Growth
Beta. Just look at it. Majestic!

What’s the longest period of time that you’ve let software that you’ve shipped stay in beta? A week? A month? A year? 

What does beta even mean in the SaaS world of continuous delivery? Is it a relic of the past?

Let’s explore further.

Showing my age

Sarge.

It’s 1999. 

In the very early hours of a Sunday morning in April I am hunched over a desk, curtains closed, in my bedroom on the first floor of my childhood home. My face is lit dimly by the glow from a CRT monitor. 

I’ve been downloading a file called Q3Test.zip for most of the last twenty four hours, praying that my 56K modem doesn’t disconnect before it finishes. 

The dialog box reads 99%. 

This file was the public beta test of Quake III Arena, which to a die-hard Quake and Quake II fan like me was like having my birthday and Christmas rolled into one. It was the first chance to play the game that I was going to be spending many, many hours mastering (or, at least, trying to).

Beta releases of games were a big deal back then. It was a chance to feel part of the inner circle.

The anticipation of games in development was primarily generated by reading physical copies of PC Zone or PC Gamer, which would be the only reason my parents could convince me to come to the supermarket. 
I would browse the screenshots of current builds, which were hyped to fever pitch by the writers. I would imagine the secretive locations in which they were being created.

After many months of following the development of a game in a magazine, news would break that there was a beta available to download, and I would scramble online to IRC chat rooms to find out where it was hosted, and whether the mirror was reliable.

Reliability was key: in the late 90’s the fastest consumer Internet connection available in my area was dual ISDN, and that was way too expensive for my family. I needed somewhere to download it from that would not kick my little modem off while it was hammered by kids with fibre connections in the Nordics. 

99.6%… come on…

Stages

Following the games industry enabled me to begin to learn about how software was made. The beta stage was usually where people like me began interacting with it, as I was an avid gamer as a teenager, and I saw it as a matter of pride to be the first at my school to play the latest releases.

Jeff Atwood outlines the stages:

  • Pre-alpha: The software is under active development and isn’t ready for consumption by users.
  • Alpha: The software is ready enough for internal testing.
  • Beta: The software is ready for external testing, but has known bugs or limitations.
  • Release Candidate: The software is almost ready for final release and only tightly scoped bug fixes are allowed.
  • Gold: It’s done to the best of our knowledge.

Note: if you’ve ever been naughty and downloaded leaks of games, the “RC” in the filename stands for Release Candidate…

Now, for betas specifically, there are closed and open betas. 

A closed beta typically involves an invite-only period featuring users that have been selected to use the software, either because the number of concurrent users needs to be controlled, or because they are friendly customers who aren’t bothered by some bugs, or both. 

Open betas are where normal people like you and me get involved. As much as I would dream of id Software inviting me to their office, unfortunately it never happened.

What about in SaaS?

Now that I’m older, I don’t work in games; I work in SaaS. 

It’s an industry that moves very fast. You could posit that the software development stages listed above seem fairly archaic compared to the mindset of shipping to production continuously.

However, the formalism of the aforementioned stages is a defense mechanism: very large software projects such as AAA games and operating systems can’t ship broken code as the consequences would be devastating. 

Either a company could be humiliated by shipping millions of copies of physical media containing a broken build, or, even worse, a poorly tested operating system could ruin a person’s computer.

But, fortunately, in SaaS, we deliver via the browser and we can ship updates whenever we want. With the expectations of the expedience of delivery being high, we continually ride a line between acceptable speed of delivery and acceptable quality

This is why beta programs are so popular: allow an increase in the acceptable speed of delivery and a reduction in the acceptable quality. Win-win, right? 

Atwood pointed out two trends in 2008 which are more true than ever ten years later:

  • The definition of beta grows more all-encompassing and elastic every year.
  • We are awfully eager to throw alpha quality code over the wall to external users and testers.

Faster, faster, faster!

A mantra from Facebook’s past. It’s now “Move Fast With Stable Infra”, which isn’t quite as catchy.

Despite some parts of the software industry still following the release stages mentioned above, SaaS and the drive towards continuous delivery has created a trend of perpetual beta. 

There are a lot of large websites that continually have functionality in beta. Sometimes the entire service is beta: Gmail famously had a beta label for many years and so did Flickr.

I could summarize the current climate that I experience in my area of SaaS as:

  1. Expedited delivery of something of value to customers is the number one priority of any development effort.
  2. Done is always better than perfect.
  3. Perceived speed of development teams is the most important trait for Engineering in competitive VC-driven marketplaces. 

The last point may be cynical, but sometimes I feel it.

In 2010, an article suggested that delivering software using agile encouraged perpetual beta. But does this incremental, speed-first completion-second delivery mean that nothing ever gets finished properly, or is it actually useful?

Usage of beta in SaaS

In my own experience, there is often a lot of confusion around what a beta release actually means, and this confusion often compounds when views of engineers are mixed with those of non-engineers. 
Here is a list of situations that I’ve seen a beta flag used:

  • Features that are still slightly buggy but generally work and deliver value.
  • Features that aren’t yet complete (i.e. have a subset of proposed functionality).
  • Features that may or may not remain in the software based on customer feedback. This includes innovative or experimental features.
  • Features that are conceptually complete but haven’t yet reached the desired level of engineering quality (e.g. scaling, monitoring, or infrastructure robustness).

If this is the case, then there can be two forces that can drive the labelling of a feature as beta:

  1. Product: The feature or product isn’t complete or useful enough for all users to find value in it.
  2. Engineering: The feature or product is not scaled or robust enough for all users to have an acceptable experience.

This is very much a trade-off between Product and Engineering, and requires both disciplines to come  together to make a call on the use of the label, and most importantly, where progress on that software needs to get to before the label can be removed. 

And yes, it should get removed.

A healthy dialogue between both sides ensures that new features get into the hands of users through beta programs early enough so that feedback can be collected, but late enough so that they won’t be disappointed with what they see. You only get one chance at a first impression.

The positive side of beta

There is an overwhelmingly positive side to being able to run beta programs, and in users being more open to using beta software, even at work.

In fact, I’ve seen the excitement that I used to experience as a teenager when gaining access to beta releases of new games manifest in some of our own users when we invite them into beta programs that we are running. (OK, I exaggerate, but some do get very excited.)

By running beta programs in SaaS, there are a number of tangible benefits:

  • They allow you to get feedback much earlier in the development process.
  • They encourage more creative and innovative ideas because the impact of failure is much lower.
  • They allow your infrastructure to go under an initial stage of real load, rather than the simulated load you have done during development.
  • They can strengthen your relationship with your most engaged customers by giving them something new and exciting to play with earlier than everyone else.

We learn a lot from our beta programs, and some of our customers give such great feedback we should probably be paying them rather than the other way round.

Abuse of beta

However, for all of the positive talk about beta software, I’ve also seen the beta label be abused. 

Here are a list of things that I feel beta software should not be, yet have witnessed:

  • Untested. I mean come on, that’s just incredibly lazy. Your users are not your QA, and you shouldn’t feel comfortable or proud giving untested software to them.
  • Finished or indefinitely paused. Sticking a beta label on something that will never be completed is an anti-pattern. The point of a beta is to learn something, and should not be used as an excuse to ship substandard or orphaned work.
  • Used to mask diffidence. There is nuanced case where the beta label is used as a proxy for being nervous or embarrassed about the quality of what is being shipped.

Whenever I’ve observed the above happening, it’s often not via the intention of the team.

The first two situations often occur when there is external pressure to ship regardless of quality, such as when a feature has been promised and needs to go out to save face. Alternatively, they can both happen when reprioritization causes a team to drop what they are doing and work on something else immediately, and the “beta” launch is the salvage from the wreck.

The last situation stems from a nervous team, and may not even be indicative of their anxiety of how their users will react, but rather how important internal people will react, such as their managers, peers or the executives. The beta label is used as a buffer for any initial disappointment with the shipped feature.

Abuse of the beta label is often not because you have a poor performing team: you may have a poor performing culture or organization. It’s worth being mindful of this happening.

To conclude

Try not to spend over 5 years in beta. This product seemed to take off, though.

Beta releases, when handled correctly, are excellent tools to ship faster, please users, generate excitement and get valuable feature and infrastructure feedback. 

However, you must always ensure that beta releases are being done for the right reason and are not being used to hide smells of poor organization or a culture that doesn’t care about the quality of the code that is being given to its customers.

Beta is for learning and experimentation. It is not an excuse for unfinished work.