Tag Archives: Estimating

This Cognitive Bias is Hurting Your Company

Imagine using Microsoft Excel to calculate your project’s budget. Now, imagine that you shared your file with another person and when they open it they get a different result than you did. How annoying would that be?

Or, imagine a scenario where you receive different answers from Excel that depended on the last application you had used? Use Photoshop and then Excel, get one answer. Look at your Outlook calendar and use Excel… get a different answer.  That would be awful. If you were unaware that the tool was producing different results, you would likely continue to use it and be surprised when you got problematic results. If you did know that an application behaved this way I think it is safe to say that you would stop using it. A tool like that would be useless. Or would it?

Your brain can anchor its judgement, even when you are not aware that it is happening.

Your brain can anchor its judgement, even when you are not aware that it is happening.

You see, our brains are such a tool. Different people can take the same set of inputs and come to completely different conclusions. Even more interestingly, the way our brains interpret new information can be profoundly influenced by other information that we have recently been exposed to. It’s possible that what is at work here is a cognitive bias called “anchoring.” Anchoring can cause your stakeholders to have unrealistic expectations, your team to have distorted estimates, and your teams to focus on the wrong things.

For this post, we will start with a short definition of anchoring, then share a classic example of anchoring. After that, there are a few examples from the world of software development. The examples will be followed by some research on the topic. Lastly, I’ll share a few ways you can try to combat anchoring.

What is Anchoring?

Anchoring is flaw in the way our brains process information. Our brains do not give equal weight to all the information it receives. We give preference to some information. Anchoring is the name given to the tendency to put too much weight on the first piece of data we receive. The mind attaches itself more strongly to the first piece of information, and doesn’t give equal consideration for all the later pieces of information it receives.

A Classic Example – Car Shopping

I was in search of a car for a teenage driver. While it might have been easier to find a car like the one from the movie Uncle Buck, we thought we’d spare him the embarrassment.  Our quest became finding a car that was safe, mechanically sound, and ideally,  a convertible. Oh, and we wanted to spend $2,500 or less. It wasn’t imperative that we stay with that budget, but it seemed that we ought be able to find a vehicle for that price.  Thus, began our quest.

We scoured the Craigslist posts for weeks. There were many listings that were close to the criteria we were seeking, but the sellers’ “asking price” was higher than our budget. I’d use a couple reference web pages, including the Kelly Blue Book and the North American Dealer Association (NADA) page in an attempt to validate the asking price. Almost everybody was asking more than the industry sites indicated. Some were asking a lot more. But, we know you have to negotiate, right? No matter how much outside information I got, my mind kept telling me that the prospective seller would need a higher amount. They were asking for more. I was anchored on the initial ask, despite having data to indicate that the ask was inappropriately high. It is hard to treat the first price equally, even in light of later information.

Anchoring on Software Teams

The car shopping example is a classic example, but doesn’t directly relate to our professional lives, unless you sell cars for a living. While anchoring can lead you to pay too much for a car, the potential downside for technology teams are orders of magnitude bigger.

1. The Dreaded SWAG

Let’s say the big boss wants a set of features brought to market. From initial high-level discussions, the work seems to be pretty simple, an initial Scientific Wild-Ass Guess (SWAG) of three months comes up during a hallway conversation with a couple folks from the technology side of the business. Everybody gets excited about the new product, and dive in to scoping and planning the work.

As the details unfold, it becomes apparent that the work is more complex than initially envisioned. The team starts to understand that it will be harder to implement, and night not fit as smoothly with he existing application framework that is being used. It starts to feel big… really big.  It starts to feel like the work is going to be three or four times as large as the SWAG  Unfortunately, the initial estimate of three months is going to be weighted more heavily than all the new, refined, information. Sure, the business needs to figure out if it’s still worth the bigger price tag. And, because of anchoring effect, it will be very challenging to reset stakeholder expectations. That initial estimate will be remembered, and it will rear its ugly head for months to come, particularly as the work (most likely) continues to grow.

2. The Story Size

We estimate work on a regular basis. Many teams use story points for estimating the size of  product backlog items There are many ways that assigning story points to backlog items. Some of the estimating methods are profoundly susceptible to anchoring. Here are two estimating patterns that create anchors when assigning story points:

  1. A small fraction of the team estimates the work and then takes it to the team to review and update.
  2. The whole team is involved in estimating the work, but as the work is being discussed, an initial number is voiced and then others are asked for their opinion.

If you do either of those techniques, please stop. No, seriously. Stop!

If you’re interested in learning much more about Story Points, please check out the Story Point articles that Mike Cohn makes available on Mountain Goat Software.

3. The Derailed Retrospective

Anchoring doesn’t just happen with numbers. Poorly structured, or worse yet, unstructured retrospectives can also get anchored.

There’s a common scenario for retrospectives where the team gets together to talk about what’s been happening. No structure, no facilitation. In scenarios like this, the first talker “wins” and skews the remainder of the retrospective.  The first topic raised sets the frame for the meeting. The group just got anchored. In this scenario, there’s a really strong possibility that the first topic raised is not the most valuable one to talk about. It might be the most recent (another bias), or something emotionally safe to discuss, or somebody’s “pet” complaint, but it’s not likely to be the most important or valuable one.

What Does the Research Say?

It’s important to go beyond personal anecdote, so let’s take a foray into the research side. There has been a lot of research on cognitive bias. And, even with the research, there is a lot that is not understood. The field of bias continues to produce interesting results.

One study that is often cited is a 1974 publication called “Judgement under Uncertainty: Heuristics and Biases” by Amos Tversky and Daniel Kahneman. In a demonstration of anchoring bias, Tversky and Kahneman asked participants spun a wheel. The wheel would give participants either a high number or a low number. Later, participants were asked to estimate the number of African countries that participated in the United Nations. And, it turns out that the number the wheel provided had a very significant impact on the estimate provided by participants. If an estimate in such a condition of uncertainty can be influenced by the result from the spin of a wheel, how much more might we get influenced by the estimate of a colleague under conditions of uncertainty?

Combating Anchoring Bias

Now that we’ve seen some examples and dipped our toe in some research, let’s look at some ways to combat the anchoring bias.

Gain Specific Knowledge

When people have concrete knowledge of a subject, they are less likely influenced by the anchoring condition. For example, if you are an expert on Africa’s participation in the United Nations, you would likely know the percent of countries that participate in the UN. Thus, the expert would not get anchored on irrelevant data. But beware. While that is true that people with specific knowledge on the topic are less susceptible to anchoring, people still tend to view themselves as less prone to anchoring than the average person. These are the result of research published in the Journal of Experimental Psychology.

So, the first mitigating strategy is to become more expert on the topic, thus giving you better information to work with.

Provide Multiple Explanations

When people fixate on a single outcome, they are prone to anchoring. An approach called “multiple-explanation” can help to de-bias the judgement. Let’s go back to one of our earlier scenarios where the project was estimated to take three months. One strategy to help de-bias the judgement would be to come up with explanations about scenarios by which the project was delivered in a much longer timeframe. For example, the work turned out to be complex, or needed to be highly scalable, or the team is distracted by the scores of other “top priorities” that the business puts on them. By going through the exercise of explaining a number of different scenarios and outcomes, the initial judgement can become less biased. For more on Multiple Explanation for Debiasing Judgment, read the research by Hirt and Markman.

Planning Poker – Show Your Cards

Anchoring, by definition, is over-weighting the first piece of information you receive. When estimating work, Planning Poker is specifically designed to withhold estimation until everybody who is estimating has formed their own opinion. Then, and only then, do all the estimates get exposed at the same time. It’s a subtle difference compared to what we discussed earlier regarding estimating. In that “anchored” scenario, a subset of people provided an estimate and asked for validation. In this scenario, it is all estimators sharing estimates at the same time. By revealing all estimates at once, anchoring is less likely.

Silent Writing for Retrospectives

The “first talker” scenario for retrospectives has a very simple alternative. Don’t start with talking! Instead, set a framework for the conversation and allow people to reflect for themselves, writing down their thoughts. Then, get people to share them. By allowing people to silently write their thoughts down, they aren’t as prone to the anchoring effect of the “first talker.”

Conclusion

Anchoring is insidious, and impacts our lives in many ways. While we may think we are not as prone to anchoring as the average person, we are. Simply being aware that anchoring is a bias is not enough to remove its influence on us. It is important to look for strategies and tactics that make anchoring less likely to bias our judgement. I hope the four strategies for debiasing stakeholder expectations, getting un-anchored estimates, and letting all voices get heard help your business and teams. I am interested in your feedback, so please leave a comment.

Specific to software, I found the book Software Estimation: Demystifying the Black Art (Developer Best Practices)to be quite valuable. This book provides a number of strategies for getting better estimates for your software project.

 

So, What’s the Point?

One of my teams been involved in a bit of a discussion as to whether or not defects in the product backlog should be sized using Story Points, just as we size stories. Mike Cohn has a very good blog about “Story Points for Bug Fixing.” As I read his post, I believe it is specific to the scenario where a team has inherited a set of defects from a prior release or a legacy product that they are going to attempt to work through in the midst of delivering new features. Mike’s recommendation is that defects be sized, much like stories would, and that the resulting completed work would count toward the team’s velocity. I agree with that approach. I believe the scenario that we’ve encountered is a different one, and thus a different approach may be more suitable. The scenario I would like to explore is one in which defects found in the current release’s earlier iterations. Should those defects be sized and be included in the team’s velocity?

Consider a scenario where the features set to be delivered consists of 100 points of work, and that the team’s average velocity is 20 point per iteration. The figure below shows what a release burndown might look like under these conditions. Prior to iteration 1, there are 100 points of work to do. At the end of iteration 1, we have completed 20 points of work with high quality and there are 80 points remaining. Finally, at the end of iteration 5 the backlog has been reduced to zero*.  The burndown in this scenario is “Backlog Points Remaining after Iteration – Scenario 1”

This burndown shows the points remaining after each iteration is complete. Each iteration, the team completes 20 points of work on stories that are in the backlog.

In another scenario, we identify defects from earlier iterations, size them, and include them in the iteration’s velocity. So, in scenario 2, each iteration still shows a velocity of 20, but the burndown does not reach zero after the five iterations are complete.

This burndown shows the points remaining after each iteration is complete. Again, the team completes 20 points of work each iteration.

Without the proper visibility to the information feeding this chart, it appears that the product manager is growing the backlog. However, if you were to decompose the velocity into its constituent points (illustrated below),  you see that in the second scenario the contribution of defects to the team’s velocity grows throughout the release cycle. The assumption that the effort put toward defects will grow is reasonable; if there are defects being introduced early on in the release and nothing changes to address the root cause, the defects will continue to be introduced and their contribution will become more and more significant. Without transparency to what made up the velocity, the team can feel that they have executed with their ‘expected velocity’ and the product owner can feel like she is getting asked to make unreasonable concessions due to quality problems. Do you think it is reasonable that the product owner would remove scope because the quality of earlier deliverables was not high?

This view shows the points that were dedicated to stories in the backlog compared to points assigned to defects.

So, what to do?

I do agree that bugs from prior releases should be sized by the team and prioritized by the product owner, much like a new story. In that case, defects from prior releases would be included in the velocity, and the product owner could determine what new features could be completed if work toward resolving legacy issues was suspended.

If defects introduced in earlier iterations are going to be fixed in subsequent iterations, their contribution to the team’s velocity should be made big and visible. Only if the information is visible will it get the proper attention so the team and product management can work to figure out the root cause of the defects. If the team is not going to make the effort put toward resolving defects visible, I would suggest that it would be best to not size the defects and to merely let the ongoing reduction in team velocity be the canary in the coal mine. Unlike the scenario that Mike spoke about, the team cannot reasonably suspend defect resolution to deliver additional features.

Lastly, if the team’s anticipated velocity includes an allocation to fixing defects that are yet to be introduced, this needs to be accounted for as part of release planning, as well.

Ultimately, it is critical to identify the root cause of the defects and to address the root cause. Without addressing the root cause, we are left with an ever-increasing allocation of time to resolving defects that we are introducing as we build new features.

Conclusion: Sizing defects that are introduced as part of the release that you are currently working on can produce a misleading velocity. If you do want to size defects and have them contribute toward the velocity, segregate the points from Defects and Stories so that the effort toward each becomes plainly visible.

Footnote:
*For the sake of illustration, I have simplified the scenario. Indeed, there is often some variability in velocity from one iteration to the next. This scenario also assumes that the product backlog is constant throughout the five iterations.