The Surprisingly Obvious Reasoning behind Scrum

Scrum is a framework for developing, delivering, and sustaining complex products. It is designed to help teams work together to incrementally create value in short periods of time. Today, it is the standard approach in developing software, but it has also found its way into many other areas.

Although I have worked with Scrum already for some time, reading Jeff Sutherland’s book (the creator of Scrum) led to many new insights. Mainly because Sutherland explains the reasoning behind the design decisions he made with Scrum. This article summarizes these insights.

First, let’s take a quick look into how Scrum works. It is based around a simple idea: In a project, regularly check if you are heading in the right direction. Is what you are doing actually creating value; is it what people want? Sutherland calls this the Inspect-and-Adapt-Cycle. It consists of four steps: Plan, do, check and act. If you make sure you have something useful after every “Do”, you basically got it. Scrum seems more complex because it consists of roles, events and artifacts. But really each of these only exists to help you to get better in either planning, doing, checking or acting.

Fast, incremental progress is central to Scrum. Naturally, those who create this progress, the team, are at its center. The team works on manageable units of work called tickets. They do that in repeating time frames called sprints. These sprints are often 1-2 weeks long. In each sprint, the team aims to finish a set amount of tickets. If a ticket meets the definition of done, this means it has incrementally improved the product. Every day, teams come together in the daily Scrum and briefly discuss what they are working on and where they might need support from other team members. At the end of each sprint, the team reflects two essential things: First, on the progress they made with the product (Review) and second, on what inhibits the team in working effectively (Retro). The Scrum master is responsible for removing these obstacles, he also keeps an eye on the process. Then the cycle starts again, with the team planning their next sprint. The product owner is responsible for the product vision. As the name suggests, he owns the product and is responsible for the outcomes. He creates new tickets, refines them with feedback from the team, and makes sure the team works on the most important tickets first. That’s it in a nutshell.

Interestingly, out of the 230 pages of the book, Sutherland describes how Scrum works on only four pages hidden in the appendix. It feels like Sutherland tries to say: “Here is what I know about people, that’s why Scrum is like it is. If you don’t like it, come to your own conclusions.”

Before we move on, let’s get some common misconceptions out of the way, and quickly talk about what Scrum is not. Scrum is not a method (and not a religion), but a framework. The idea of Scrum is not to plug another layer of bureaucracy on top of existing ones, but to give guidance to making results faster.

Also, Scrum does not relieve you from having an idea of what your product will be. Quite the opposite, it is essential to have a product vision. You can update this vision throughout the process, but working iteratively and user-centered is not an excuse for not knowing what you want.

Scrum just urges you to check frequently if what you are doing is actually helping you to achieve your vision. That is why, I believe, that understanding the ideas behind Scrum is important. It is not a recipe for success, but a handrail towards reaching it.

The Alternative to Scrum

To Sutherland, the opposite of Scrum is the waterfall approach. Here, every piece of a project is mapped out in detail. All requirements, exact project specifications, starting and end dates and all dependencies for all tasks are laid out in detail before anybody starts doing anything.

This approach is in an obvious way inflexible. It becomes a problem when the plans don’t account for newly emerging, complex scenarios. Changing the plan, would waste all the work you did earlier, so it isn’t allowed. The tragic part is, that everyone knows better. There are many smart people involved in executing the meticulously prepared plan, that are aware that what they are doing might be of no value. Waterfall projects are resistant to change by design.

This robustness is not always a bad thing. If you have highly standardized and repetitive work, it might be good to operate in such a fashion. But as tasks that follow a simple and repeating pattern often can be automated, projects which require flexibility become the norm. Especially for teams involved in crafting something new, where requirements are not super obvious, flexibility is essential. Sutherland gives examples of houses being built, schools being run, and governmental programs being realized, all using Scrum.

With the waterfall out of the way, let’s move on to the centerpiece of Scrum, the teams.

Teams

Scrum is centered around teams and puts much effort into them working well together. With good reason, while the difference between the fastest and slowest individual is roughly ten-fold, the difference between the fastest and slowest team is two-thousand-fold. Improving a team has orders of magnitude greater impact than hiring one individual high performer. Further, even if your teams only consisted of high performers, if they don’t work well together, all their potential is wasted.

Today, businesses still mostly focus on individuals, even if we can only find very few things that aren’t a team effort. Think of performance bonuses, promotions or hiring. Everything is focused on the individual actor. Even worse, organizational structures often encourage competition between team members. In many organizations, careers are made by rising up the hierarchy. As the number of promotions is limited, the incentive is to make yourself look good compared to everybody else and not to collaborate in the best possible way.

The question remains: What makes a team great? The paper that inspired Sutherland to create Scrum, tried to answer exactly this question. It looked at the product development teams of engines, printers, cameras and early computers by Fuji, Canon and Honda. The paper found three characteristics of great teams:

Autonomous. Teams are self-organizing and self-managing. I talked about how focus on individuals create the wrong incentives, autonomy is a way to fix this problem. It is left to the members of the team to decide how to reach goals. The (only) task of the leaders/managers is to provide guidance, money and moral support. Autonomy is about transferring power to the team, this shifts incentives from pleasing higher ups to creating value.

Transcendent. Having a sense of purpose and a desire to be great. Teams start at the guidelines set by management, but great teams quickly develop their own aspirations. Transcendence is about the team as a whole wanting to become better.

Cross-Functional. Teams should have all the skills they need, to actually release a product increment by themselves. If teams are build this way, organizational divisions suddenly form around products and not around functions.

Keep Teams Small

For well functioning teams, communication is key. In Scrum, teams hold a daily meeting. It should be short, no more than 15 minutes, and helps to keep everybody up to date. Only with frequent exchange, team members can support each other with their expertise. The idea is to get the most actionable and valuable information in the least amount of time.

If you want your team to communicate closely, you have to keep teams small. For a simple reason: With rising number of team members, communication channels rise exponentially. A team with seven people has 21 communication channels, a team with 14 people already 91. Research has shown the optimal size for teams to be 7 ± 2. Teams of more than 9 people become slower, or they internally divide into subteams. That’s why it often doesn’t help to add work-power to a project already behind schedule. Brook’s Law states, when work-power is added in a late state of a project, the project becomes even later.

Laziness isn’t a Characteristic

Working in teams is not easy. We all had feelings of unequal engagement and were upset about others making “stupid” decisions, or being jerks. In these situations, we unfortunately often experience the “Fundamental Attribution Error”. It goes like this: When we explain the actions of others, we think they are motivated by their character. But when we explain our own actions, we find ourselves responding to a situation. We might say, I won’t work hard because I won’t get a bonus (reaction to situation), but when talking about others, we might say, he didn’t finish his work, because he is a lazy person (characteristic). We overestimate the influence of a person’s characteristic and underestimate the influence of the environment on their actions.

“It’s the systems that surround us, rather than any intrinsic quality, that accounts for the vast majority of our behavior.” — Sutherland

The knowledge about the fundamental attribution error also helps us to solve a dispute among managers. There are two theories of human motivation (by Douglas McGregor): The first, theory X, says people are by default lazy, stupid and work only for money, while theory Y assumes people are internally motivated, like their job and like getting better. While assuming theory X, managers must punish and micromanage. Assuming theory Y, managers see their workforce as a valuable asset that needs nurturing. The attribution error shows us, that neither is true. Basing a management on assumptions of intrinsic behaviors makes little sense if the environment is the largest driver of action (this then is known as theory Z). Scrum acknowledges this fact and tries to leverage it. Instead of blaming individuals, it tries to fix the system, that’s what the sprint retrospective is all about. It urges teams to reflect on why they work the way they do and enables them to change environments that make them unproductive.

Don’t Worry, Be Happy!

Happiness is a predictor for success. I found it surprising, that it isn’t the other way around, but it makes sense when thinking about it. Work can bring a great amount of joy. It can create a pleasant state of flow and a sense of purpose. These things all influence how good you work and the quality you produce. Sutherland quotes a few studies that show, people who are happy actually create stuff faster and with better quality. Scrum continuously tries to improve happiness by removing obstacles.

“Happiness leads to success in nearly every domain of our lives, including marriage, health, friendship, community involvement, creativity, and, in particular, our jobs, careers, and businesses.” — Sutherland

Also, it enforces a sense of accomplishment. In the sprint review, teams look back on the work they finished in the last sprint. They get a sense of progress and realize how they got better and faster compared to previous sprints. The idea is that becoming faster and creating more value leads to more fulfillment, sense of purpose and in turn to more happiness. Happier teams work better together, and in turn become faster and more productive.

A “danger” of Scrum is, that early improvements feel so good that people stop to strive for further improvements. Sutherland calls this the happiness bubble. The Scrum master has the responsibility to get teams to realize areas of further improvement. He is around to ask the hard questions and to pop the bubble.

Planning and Doing

Scrum acknowledges that the future is hard (often very hard) to predict. Planning complex projects is difficult, because crucial aspects of a problem are not apparent straight away; they get lost in complexity. Sometimes it is surprisingly hard to a priori say how long a complex task will take. Scrum acknowledges that the larger a project is, the harder it is to determine its complexity.

That’s why work in Scrum is always done in manageable, well-defined pieces called tickets. Thinking in tickets is a way to manage complexity. The size and complexity of a ticket is estimated by using a relative measure of effort called story points. Typically, there are only a few values to choose from: 1, 2, 3, 5, 8, 13, 20, 40, 100. The limited values reflect the uncertainty in estimating large tasks. If a ticket is too large, say larger than 13 points, it needs to be broken down into smaller, more manageable pieces. This ensures tickets never exceed a certain complexity threshold and thus can be finished in a foreseeable time horizon.

For something to be considered finished, it needs to meet the definition of done. This definition of done is a set of properties, the team decides on. For example, the definition of done for this blog entry is: I must have read it aloud, somebody else should have read it, and it must appear on my website.

The definition of done is valuable because it gives you clear guidance. No longer is there the state of “it’s basically done, but I still need to do this one tiny thing”. Either it meets the definition of done or it’s not done. Further, creating a definition of done gives you an opportunity to bake quality control into your work. Requiring your work to be peer-reviewed in order to count as done is an example of this.

Waste

There is nothing more frustrating than waste. Working hard and seeing your efforts not being useful in any way is devastating. According to Sutherland, one of the most wasteful things you can do is multitask.

Studies show, nobody can multitask well. People who believe, that they can multitask well, are actually worse than average. When multitasking, you waste a lot of time in context switching. Thus, Sutherland advises: Do one thing at a time.

“Waste is a crime against society more than a business loss.” — Taiichi Ohno, the “father” of the Toyota production system

Another area where you can avoid waste, is doing things right the first time. Fixing a problem or bug (in software development) later takes up to twenty times as long compared to when you would have done it right away. Many things are impossible to do right the first time, but errors that arise because people are unfocused or overworked can be avoided. This is the reason why sometimes working less is faster. To be productive, it seems, not working is as important, as working.

Priorities

Let’s consider a project that has 10 tasks that each take 10 days to finish. At day 90 we could either have 9 tasks finished, or we could have 10 tasks, each 90% complete. Now an unexpected event happens, and we have to finish that same day. In the first case, we have a product that is usable and has 90% of features, while in the second case we have nothing. Doing half of something is essentially doing nothing. Being done, is where the actual value is created.

This gets even better, once we start prioritizing our tasks. It’s obvious to do the thing that creates the most value first and the least value last. The question of what creates the most value should be informed by the product vision. Coming back to our example: If we had to deliver early, we not only had a product that is 90% usable but also with 90% of the most important stuff. Maybe in the end, the last 10% didn’t matter in the first place. If you hit a hard deadline before you are completely finished, you at least are sure that the $n$ most valuable items are completed. And that in turn you really have something that delivers value.

Even better, you can stop early by decision. If you at any point realize 80% of the value is enough, fine. Because every ticket gives you a fully functioning increment, you can just stop. This saves time, money.

Closing Thoughts

People often come into contact with Scrum because somebody in their company heard it somewhere, and suddenly everybody has a new “role”, but still does the same. It’s a shame that Scrum has become this buzzword, because at its core, it is a critique of and a reaction to traditional ways of working. Without understanding the Why behind Scrum, it makes little sense to pay a Scrum master. If you don’t understand what Scrum is trying to fix, how can you expect improvements?

If you liked this summary, give “Scrum: The Art of Doing Twice the Work in Half the Time” a read.

Thank to Jordan Job for creating the scrum graphic. Check it out here