Facts!

If you’re a programmer, you’ve probably relished in some of the many Jeff Dean facts. Here’s a selection for you:

• Jeff Dean’s PIN is the last 4 digits of pi.
• Jeff Dean proved that P = NP when he solved all NP problems in polynomial time on a whiteboard.
• Jeff Dean once shifted a bit so hard it ended up on another computer.
• Jeff Dean can beat you at Connect 4 in three moves.
• There is no ‘Ctrl’ key on Jeff Dean’s keyboard. Jeff Dean is always in control.

Indeed, these “facts” aren’t facts at all, and there’s plenty more where they came from. If Chuck Norris is the embodiment of machismo then Jeff Dean has evolved into the industry’s manifestation of superhuman computer science wizardry.  

You may not know who Jeff Dean is, which, I must add, is entirely unreasonable. Let’s bring you up to speed. Aside from Jeff’s primary job, which is being the Internet, what else has he done in his career?

jeff.describe()

Before he joined Google in 1999, Jeff attained his Ph.D. from the University of Washington. His research focused on optimizing compilers. His research took him to Digital Equipment Corporation’s (DEC) Western Research Lab, before being convinced to come and join a young company with some interesting problems to solve. That company was Google. According to Slate, there were only around 20 other staff there at the time. It was very early doors at a pivotal company.

During the last 19 years, Jeff has co-designed and implemented a large proportion of the most impactful and world-renowned Google infrastructure, including numerous iterations of the crawling and search platform, the initial development of AdSense, Protocol Buffers, GFS, MapReduce, BigTable (which you may use as HBase), Google Translate, and more recently, TensorFlow. If you’re working in a software company solving problems with large amounts of data, you’re probably using a system or library that evolved from something that Jeff created.

Work

It’s no accident that Jeff has been involved in Google’s most innovative projects. In addition to being a programming sorcerer, he is equally good at helping others grow through his mentorship and technical talks, thus allowing others to multiply their careers by working with him. He also is highly respected and listened to by Google’s higher ups, making sure that he gets to place himself on the most impactful projects.

A candid Quora answer was written about what it’s like working with Jeff, which, surprisingly, isn’t all roses:

I worked with Jeff Dean for about a year when I was on the Google Plus team.

He’s mostly just a really friendly, smart, helpful coworker.  Like any other Google engineer, you could walk over to his desk, ask why a particular product worked the way it did and he’d give you a really good description of the decision factors and thought process that went into that particular architecture. He gave tech talks and demos to show his work and fielded questions just like any other senior engineering staff member.

But he was also a little intimidating. Jeff definitely has the ear of upper management and they give him free rein to pretty much work on whatever he wants to.  So if you’re working on a particular project that is at cross-purposes with something that he wants to do, then you kind of just get railroaded because his code is getting *launched* no matter what…  So you may have to just adjust your plans to accommodate!

So there’s a bit of “jujitsu” to working with high-powered folks like him — you find ways to align your projects so that their weight is being thrown in behind you, not against you.

Partnership

A recent New Yorker article gave a candid insight into Jeff’s life and also his partnership with Sanjay Ghemawat. The pair work inseparably, often choosing to pair program together, bouncing ideas off each other whilst one forges ahead on the keyboard. They worked together in a similar fashion at DEC. They even live inseparably: “Uncle Sanjay” often comes to dinner at the Dean’s household on Fridays.

To the purist programmer, and aspiring technical wizard, their situation seems perfect: two best friends working on the most interesting problems, together, in a trusted self-directed manner, free from the pressure of typical feature delivery and sprints. They’re some of the world’s best engineers, so they just get on with what naturally comes next, since they are implicitly drawn to the problems that are on the cusp of innovation that will be impactful for Google.

The reality for the 99%

But, as you’re reading this, you’re probably thinking that’s all well and good for Jeff and Sanjay, superstar engineers who have actually changed the world: they can have their freedom, but we can’t. They were in the right place with the right intellects at the right time.

Instead, us less-than-geniuses get our silly deadlines set by people who don’t know anything about engineering, we get processes and ceremonies that detract from actually getting good work done, and we sit in crowded open plan offices where we constantly hear noisy sales calls on the phone all day. Good for them, we say. Good for them.

Here’s the question: are Jeff and Sanjay a direct consequence of right brain, right place, right time, or is any of their situation replicable within our own working lives? After all, their situation seems unreachable. The New Yorker describes the leveling system within Google’s engineering department as follows:

Today, Google’s engineers exist in a Great Chain of Being that begins at Level 1. At the bottom are the I.T. support staff. Level 2s are fresh out of college; Level 3s often have master’s degrees. Getting to Level 4 takes several years, or a Ph.D. Most progression stops at Level 5. Level 6 engineers—the top ten per cent—are so capable that they could be said to be the reason a project succeeds; Level 7s are Level 6s with a long track record. Principal Engineers, the Level 8s, are associated with a major product or piece of infrastructure. Distinguished Engineers, the Level 9s, are spoken of with reverence. To become a Google Fellow, a Level 10, is to win an honor that will follow you for life. Google Fellows are usually the world’s leading experts in their fields. Jeff and Sanjay are Google Senior Fellows—the company’s first and only Level 11s.

Here’s another thought: if 31 year old Jeff and 33 year old Sanjay turned up at Google today, would they be given the same freedom and autonomy that they currently have at 51 and 53? Or would they end up in a regular development team for countless years, working on a tiny, tiny cog in a gigantic machine?

A formula for success

We could make some hypotheses about why Jeff and Sanjay have been so successful.

• Preparation. They are both extremely smart individuals.
• Luck. They joined a world-class company at a critical time in its growth, allowing them to work on the most important problems.
• Riding the company growth curve. Their success in said problems allowed them more and more access to two critical things: the ear of the upper management and the visibility of the cutting edge of their domain.
• Earned autonomy. Trust from management allowed them their own choice of work and protection from less innovative projects taking their time.
• Multiplied output: Access to innovative work allowed them to apply themselves to the problems that made themselves and the company grow.

Not all of us have the fortune or geographical placement to get a foot in the door at the very beginning of the next Google or Facebook. However, there are ways that we can create opportunities for our staff to emulate how Jeff and Sanjay work once we get our departments to a reasonable size: say to the point that there are a small handful of engineering teams.

What we can do is to add roles to the top of the individual contributor (IC) track that allow staff to become more self-directed, to propose their own projects, and to live outside of traditional teams.

The top of the IC track

A year ago at Brandwatch we promoted a small number of senior and long tenured staff to the position of Principal Engineer. This isn’t a role that we have invented; a quick search on Google will show you that it’s pretty well known and has been around for a fairly long time. For example, Jay Kreps was a Principal Engineer at LinkedIn when they were working internally on Kafka. There are similar role names at similarly large companies, such as Distinguished Engineer.

Engineers in these roles aren’t just hitting deadlines by clearing out tickets. Instead, as described by Drew Eckhardt, they’re doing work that materially improves whole company gross profit. This can be innovative architecture that leads to new products, scaling and optimization that saves significant percentages of company budget, and acting as a role model and coach to other engineers who become dramatically better at their job for spending time with them.

From my research, the role at companies of that size can often include management. An answer on Quora suggests that people at this level at Google (Level 8+) have more than 50 people in their division. However, we are not at Google size, nor did we want to blur our individual contributor and management tracks together.

We kept our Principal Engineer role solely about the engineering, because that’s probably what Jeff would have wanted if I could hire him.

How I implemented a Principal Engineer role

We have an Engineering department of around 160 people. We wanted to allow the chance for our best performing senior engineers to break away from the traditional scrum team format and to become more self-directed in their work. The exact reporting line differs in our numerous divisions, but I wanted to share with you how it works in my division.

I run the Applications division, which is mostly focussed around a suite of our core SaaS products. This does involve a number of interesting infrastructure and architecture challenges, but the bulk of the development effort is closer to the user: much more so than the division working on our web crawling, data infrastructure and primary storage.

Due to this, the roadmap of my division is primarily led by Product rather than Engineering. Carving out the space in order to do Engineering-led innovative projects – notably those that may not have a immediate pay off, such as infrastructure that can invent new product possibilities – can be hard to prioritize against Product’s feature roadmap.

To reuse a compiler optimization term, I hoisted the promoted Principal Engineer out of a team and had them report to me instead.

The Principal Engineer’s own work stream is set mutually between myself and them. This sometimes means that they are working alone, and sometimes they may be doing some work for one or more teams. This allows them to be a welcome addition to any given project, rather than a spoken-for resource that could do any old piece of work dished out to them. It protects their time and their interests.

A recent article from Silvia Botros, a Principal Engineer at SendGrid, and the accompanying comments on Hacker News highlight how the Principal role requires cross-organizational collaboration, whole department and company understanding, and strategic thinking about where technology is going, both inside and outside of the company. This aspect is the key that sets Principal Engineers apart from their Senior Engineer counterparts.

Some of the traits that we outlined for Principal Engineer are below. These traits form part of our internal career tracks document.

• Experience: Typically has 10+ years of experience. May have previously been a Senior Engineer, Principal Engineer, or CTO at a start-up. Is a deep technical expert in their area and has a drive and vision for where we should be going in the future.
• Technical knowledge: An expert, potentially at a global level, in their area. Designs, builds and suggests architecture that will grow the company in the future. Has a track record of leading the design and implementation of high risk or mission critical projects, particularly those that require the application or invention of new technology or techniques.
• Mentorship: The go-to mentor and confidante for our engineers. Is an expert and patient teacher.
• Consensus building: Champions new technologies and the modernization of legacy systems. Is able to win the hearts and minds of senior leadership in the company and also motivate and drive their colleagues in Engineering.
• Conflict resolution: Can contribute to discussions and even resolve conflict at a senior leadership level. Rides the balance of business needs and engineering needs expertly.
• Time spent reading or writing code: 75%+

Currently, our three core products have one Principal Engineer mapped to each of them. As far as I can tell they are happy in their roles, but I’m sure they can correct me in the comments if they’re not.

It’s working out pretty well so far. The Principal Engineer in my division has delivered a new distributed bitmap index system to speed up our products. In another other division, we led a major project to rearchitect our storage to deliver 50x speed ups for our customers with the largest data sets.

In summary

You and I will probably never be as successful as Jeff Dean or Sanjay Ghemawat, however, we can create the space within our organizations to allow others to form similar skills and work on highly interesting and challenging problems.

Implementing Principal Engineer roles, even at smaller companies, can have an extremely positive effect: it can champion engineering-led projects, give individuals great career growth, inspire others to stay on the IC track rather than jumping into management for progression, and, most importantly, can result in the invention of some excellent technology.

Who could be your Jeff Dean? Think about it.

Howl

It’s Sunday morning as I write this. I’ve just returned from walking the dog in a strong gale. That same wind is now squeezing itself through the rubber seals around the skylight above me. It sounds like I am on a boat.

This desk that I am writing from has been in more frequent use recently. Over the last few months, there have been numerous changes at work that are making us a more remote-friendly department.

• My manager is now remote (-5 hours) as we have been through a merger.
• I now have two of the teams in my division located in Germany (+1 hour).
• I’m collaborating on projects where most of the discussion is via Slack and video call.
• We are encouraging our staff to ask for the flexibility they desire to make their lives better.

Now, I am aware that there are many of you that work remotely, or have had flexibility in their work for an exceedingly long time.

However, this transition is new to us, and new to me as well. In earlier stages of the company, our founding CTO was staunchly against staff not being in the office. Perhaps that was the correct attitude to have when the future of the business was still uncertain.

The task at hand now, although still time critical and important, has become more of a marathon than a sprint. We’re also much bigger.

At the time of writing, the company is around 550 staff globally, and working with colleagues across time zones is the norm. This, plus the increased flexibility in our work days – which I believe is as important for retaining staff as good pay – means that physical proximity is becoming less important.

Life and happiness

Around the time of the merger, we moved into a bigger house further away from the centre of the city, and I now have a proper space in which to work comfortably. I’m very productive here.

Initially, after shifting my meetings around, I created a meeting-free Friday which I worked from home. I’ve now done a similar thing with Tuesdays, meaning that I have three fairly “shallow” days at work, mostly filled with meetings, coupled with two “deep” days at home where I make real progress on projects.

Before I spend the rest of the article highlighting the hard parts of this arrangement, let me say that this is one of the best things that I’ve done with my routine:

• I spend 2 hours less a week commuting.
• I feel happier.
• I am definitely more productive by ring fencing my time.
• I appreciate the office and my colleagues more when I am there.
• I have a better understanding of the common frustrations of not being physically present in the HQ office.

But, adjusting has been difficult, and I wanted to write about the challenges of being away from the office which have been both physical and mental.

Physical considerations

Let’s get the physical challenges out of the way first, because in some way they are the easiest to remedy.

Set up

Firstly, if you’re going to be working from home with regularity, you need to make sure that you have a place to work that is comfortable and ergonomic. Working from home shouldn’t mean hunching over the coffee table on your laptop.

If you want to look after your body, and especially your arms and hands, you’ll need a desk at the correct height, and a comfortable chair that ensures that your eyes are aligned with the right part of your screen. You can use an ergonomics calculator for this.

And don’t brush it off as something that won’t happen to you: RSI is very real. You might not experience any for years, but it’s challenging and frustrating to reverse if you do get it. I never use laptop keyboards whenever I can help it. Any extended period of typing is done sitting at a desk with a keyboard.

When I started working from home regularly, my home desk had a stool which had been fine for light occasional use. But after a whole day on the stool my back and shoulders were aching tremendously, so we’ve recently invested in a high quality chair. The last couple of weeks have been extremely comfortable.

Making sure you move

When you commute to work, you forget that you do a lot of moving: from your bed to your mode of transport, to some destination hub, to the office, and back again. While you’re in the office, you’re walking around from your desk to the kitchen, to meeting rooms, and you’re probably popping out for a coffee at some point during the day, and walking out to get some lunch too. All of those steps add up.

On my first day working from home, without realizing, I didn’t leave the house. Apart from visiting the bathroom and the kitchen a few times, I’d been completely idle. I’d probably walked less than a hundred steps.

Therefore you need to ensure that you’re doing enough physical activity. Perhaps you could use the period of the day where you usually commute to fit in walking or running, and do the same at lunch time. If you don’t, then you can end up feeling stagnant and house-bound, which isn’t good for your physical or mental health.

On to that.

Mental considerations

The physical considerations are important, but not as important as the mental ones.

Don’t regress

I spent a lot of time on a computer as a teenager (is it any wonder I ended up in technology?) and I vividly remember the feeling of having spent yet another day of the summer vacation glued to the screen, whether to play video games, spend hours chatting on IRC, or just twiddling my thumbs while the 56K modem loaded pages slowly.

As I eventually rolled into bed, I felt lethargic yet wired from drinking too many soft drinks, and I was gradually becoming more unfit. Yes, the room also smelled like teenage boy. Yuck.

What I’m trying to say is that you shouldn’t let working from home make you regress into sitting around in your pants because people can only see your top half on video call. That might be what you think you want, but it’s not what you need.

The mind plays tricks

A combination of worry, guilt and not looking after yourself properly can cause issues. You’ll need to:

• Ensure that you have healthy habits for your mind.
• Deal with how you think others are perceiving you.
• Battle FOMO.
• Reason with guilt from all angles.
• Create a separation between work and life.

Let’s look at each of these in turn.

Creating healthy habits

I’ve mentioned the pants anecdote in jest, and I mean British pants, not American pants. But you need to create healthy habits when working from home in order to look after your mental health. Make your working from home day closer to what you do when you go into the office, rather than what a teenager does over the summer holiday.

Most of these habits are simple:

• Get up and go to bed at the usual time.
• Get outside, multiple times a day. Go for a walk before and after work.
• Take a lunch break, and again, go outside.
• Make sure you’re eating a proper breakfast and lunch and not just snacking on whatever you find in the cupboard.
• Take regular screen breaks and stretch your body and legs.
• Do some exercise to divide the day into chunks. A 20 minute run can reset your brain.

Working from home can make you not look after yourself. You need to make sure that you do.

Ignoring how you think others may perceive you

When you’re at home, the anxiety can creep in. What are your colleagues thinking that you’re up to when they see that your desk is empty? Do they really think that you’re working from home because you’re slacking off, rather than working more deeply and productively than you can in the office?

It’s hard. These thoughts may rise and fall throughout the day. They are often just figments of your imagination. We are going through a culture shift. We are not used to it.

This shift towards a flexible working culture does not happen overnight. It takes time. As each default face to face meeting begins to turn into a Slack interaction or video call, and as each instinct to walk over to a desk and interrupt someone turns into a message left asynchronously, people will adapt.

Everyone will need to adapt in our industry or they will miss out on hiring talented people who just don’t happen to live near their office.

Bad attitudes towards remote work will change.

People who think that those that work from home, or those that work shifted hours, are in some way lazy or not “hustling as hard” as those who boast about getting into the office at 7AM and going home at 8PM leaving their partner to pick up the pieces, will eventually become remnants of a time when we didn’t use technology and trust to their full advantage.

These people are part of the past. Forget about them, because forward-thinking workplaces already are.

You are a smart person. You know how productive you are and what makes you work at your best. Do what’s right for you and it will follow that you will do what is right for the company.

Embrace your flexibility and be proud it makes you a better employee. Let your work do the talking.

Battling FOMO

Regardless of how used to flexible working a company is, when you put humans together in the same place, they talk. Humans love talking. This means that you will miss out on a number of interesting impromptu conversations that happen around the coffee machine.

I try to deal with this by simulating these impromptu discussions via Slack. If I haven’t had a chat with someone working on orthogonal projects of interest for a while, I’ll just drop them a message to ask how they’re doing and if there’s anything I can help with. See if you can lurk in more Slack channels than you actually contribute towards so you can browse the day to day conversation.

For those that work mostly or completely remotely to one another, a technique that I’ve seen my colleagues use is a meeting that’s dropped in as a virtual coffee. It’s 30 minutes on a video call with no agenda – and preferably a coffee – and allows both of your minds to wander and discuss items that you’re working on.

So yes, you’ll need to work a bit harder to keep abreast of developments. But it’s easy to do so if you’re proactive.

Dealing with situational guilt

The guilt is real. There are plenty of articles about this subject. There are numerous types of guilt that I have experienced – and still regularly experience – when being at home. Your mileage may vary here, but here are some that come to mind.

• The guilt of being at home but not doing any errands. You’re at home, your partner isn’t. You’re busy, but there is a pile of dirty laundry, dishes to do from last night, and you know there’s that stuff you didn’t tidy up yesterday. It sits there in the back of your mind. When my partner comes home to find it all left as it was this morning, what is she going to think of me? (Fortunately for me, nothing.)
• The guilt of the privilege of your situation compared to others. I think about people who are doing difficult jobs that may never get the option to work from home: teachers, cleaners, builders, you name it. A circular argument commences reassuring myself that I am completely within my means to use this privilege, which is followed by a counter argument of whether we all just have it too easy in this sector. It spins.
• The guilt of doing anything non-work related. If I use my lunch break to pick up the groceries, is that unfair because my colleagues in the office are unable to do this? What if I did actually whip the vacuum round, or get on top of the washing in-between working? Is this a breach of some unwritten contract? More guilt.

Depending on how used you are to remote working, the above may seem ridiculous or familiar.

You just need to ensure that you’re being productive and doing a good job. If you use other slices of spare time during the day to get some errands done, then good for you: plenty of people stand around chatting in the office, or play foosball, or get distracted. It’s fine. It all nets out.

Creating separation between work and life

There’s a healthy habit that is important enough to have its own section: knowing how to draw the line between the your home life and the working day.

I am writing this blog post, which is very much a leisure time activity, from the same desk and location that I use for working from home. This has negative effects:

• Work and life can blur: habits can form that mean you end up checking work email and Slack out of hours, because you’re physically in the same location. This has happened to me more frequently recently as my US colleagues send me messages that arrive during my evening time, and I often lack the willpower to not check them.
• Physical locations can blur: Residual mental stress from the workday can begin to associate itself with an area of your home that should be for leisure. Going back to that desk to do something fun, like edit photos or tinker, can make you remember that frustrating meeting you had there earlier.

I wish that I could propose some kind of solution to this, other than having a home big enough that you can have a dedicated office that you only use for work and not leisure, but in reality, I don’t have a remedy.

The best you can do is try to keep your work between set hours, and mentally check out after that.

Close all of those windows and tabs and go and do something else. Close all of the work-related applications on your phone. Tell yourself that you’re done and get away from it.

Far away.

In summary

If you’ve not spent much time working from home, then it might appear to be a privilege with little ill effect. It certainly is a privilege, and one that I am very grateful for, but it isn’t for everyone, nor is it one without downsides.

You’ll need to learn how to balance work and home life, and try your best to prevent them from blurring together. You’ll have to work harder to stay on top of information that propagates in the physical world. Most importantly, you’ll need to be careful about looking after yourself.

Decades of working in offices takes a long time to unravel from, so take it one step at a time.