Essential Scrum Metrics

Some of my most rewarding professional experiences have been in leading teams through Agile transformations, adopting scrum. I've guided not only engineering teams through this transition, but even business and operations groups such as the archivists at a major media company. Very early at a publishing company a supervisor told me that "we won't do Agile" because people with our brands "need deadlines." He believed that Agile meant 'loosey-goosey' and inefficient. He also believed that we wouldn't get good metrics from a scrum team. That's definitely not the case, and in this piece I'll provide a brief overview of 3 metrics that I believe should be observed and reported.

First, the sprint and release burndown will provide a view of the team's progress at a glance. This chart is a representation of the effort remaining over a period of time, and if updated on a daily basis it can help the scrum master to predict whether the team will be able to achieve their sprint goals. As you can see in the example below, along the horizontal axis is displayed each spring the team has in a given release. On the vertical axis we have the sum of all effort of all active backlog items at the start of each sprint. The remaining effort decreases over time (at least it should!) as the backlog items are moved to 'done.' In terms of the effort indicated on the horizontal, that's decided by how the scrum team has decided to measure it. My preference is in story points, although it can be 'size' or hours. I really don't like to use hours, because a big reason to use sizes or story points is to get away from inaccurate time estimates.

via Microsoft

Second we have sprint velocity, which is my favorite to look at. All of my teams have used story points following the fibonacci sequence. The cap is either 8 or 13, depending on what makes sense to the team, although 8 is usually what I see most often as the limit. The total of all story points committed to in a sprint is counted, and at the end of the sprint we compare the completed points versus committed. If a team committed to a total of 91 points but completed 73, that's an 80% completion rate for the sprint. Still nothing to be alarmed at, as of yet. In the retrospective the team might identify causes, such as unexpected illness of a teammate, an emergency situation that divided the teams focus, or something else. After a few sprints you can take the average of the total completed over time and find the 'sweet spot.' For example, if the team completed 73, 79, 68, and 81 story points, we could suggest that the team's current capacity is right around 75. The scrum master should remind the team of this before they commit and kick off the next sprint. Over time the team will learn what it can do and should begin planning more accurately, and the percent complete will ideally approach 100.

via Microsoft

A third area to track is scope change. This is the number of stories added to a project during a given release, and it's unfortunately overlooked quite often. In a perfect world the team would know everything it had to do to complete a project before work begins, but that's just a fantasy. Plans change, and a part of being 'Agile' is having the ability to adapt quickly to what's being learned. Keep in mind that scope change is when existing plans change, while scope creep happens when stakeholders slip in additional features or requests. Excessive scope change (or scope creep) can delay a release. A good Agile tracking system will offer a way to keep an eye on changing scope.

There are certainly other factors to keep in mind, particularly when reporting Scrum metrics. One is to evaluate team capacity. The scrum team should already be doing this during planning of each sprint, as they sort out who will be out of office and consider their average capacity, as discussed above under sprint velocity. If the team is struggling to deliver on time, even at full capacity, then either expectations need to be lowered, or more members added to the team. The challenge with adding new people to the project is the time and effort it takes to bring them up to speed. That alone can easily create a slowdown in the short term, but if it's a long-term project the benefits will be seen in the long run. 

Another, final item to watch is escaped defects. These are bugs that were missed by the team and quality assurance, making it into production. They can create frustration in end users, lowering customer satisfaction. If the number of bugs in production trends higher, then development and quality assurance processes need to be scrutinized, and the sooner the better. 

This was a simple 'Scrum Metrics 101,' and I hope you've found it useful.