How fast does your organization move from recognizing the need for change to executing that change?
Knowing the answer to that question, and understanding the metrics that help to answer it, are the keys to knowing how well your company responds to change. This is especially important for companies that have adopted agile methodologies into their development processes.
There are a variety of metrics that program managers can present to executives to illustrate how well the organization is responding to change and adopting agile practices. These metrics can be divided into two groups: value metrics and productivity metrics. Both sets of metrics are vital pieces of information for understanding an organization’s agility.
Value metrics measure the value that an organization is delivering to its customers. These metrics are both internal and external measurements that provide insightful information on the organization’s ability to deliver value through agile development processes.
Slack time allows employees to stretch their minds and look for creative ways to improve and innovate. Anthony Mersino, PMP and agile transformation coach, says high-performing teams are only able to maintain their level of performance through slack time, which allows them to focus on improving processes, tools and knowledge. It is an invaluable tool that speaks to an organization’s investment in continuous growth, a key tenet of agile.
In addition to measuring the actual amount of slack time project teams have, Tim Ottinger, agile coach and veteran software developer, suggests two ways to collect information on the use of that time:
- Survey how much time was spent on learning and fixing skills/processes.
- Identify what changes the team made and what the results of those changes were.
Net Promoter Score
A company’s net promoter score (NPS) is a powerful way to measure customer sentiment and satisfaction with a company’s performance. Organizations gather this data through customer surveys that include a question similar to, “On a scale of 0 to 10, how likely are you to recommend [company name] to a friend or colleague?” Those who answer 9 or 10 are promoters, those who answer 7 or 8 are passives, and those who answer 0-6 are detractors.
To calculate the NPS, take the percentage of promoters and subtract the percentage of detractors. The number itself is useful, but follow-up questions can add another layer of insights, especially if a score is low. Asking questions such as “Why did you give us that score?” or “What can we do to improve?” can deliver valuable customer insights.
The NPS speaks directly to agility in two ways: It provides direction for changes and improvements, and it tracks changes over time. By knowing how many customers are promoters, how this number has changed over time and what suggested improvements customers identify, companies get an insightful look at what value they are delivering to customers.
Team Member Happiness
Happier people are more productive — but measuring happiness is not about measuring productivity. Because a key focus of agile is teamwork and collaboration, this is a valuable metric for executives when they are measuring the effectiveness of the company’s agile processes.
If team members are happy, then the processes in place are working. Therefore, a warning sign that processes are not working is a dip in happiness.
A simple survey can give organizations insight into team member happiness. Similar to the NPS question, simply asking, “On a scale of 1 to 5, how happy are you with your team (or organization)?” will provide an indication of happiness.
Where value metrics provide insights on the value driven through agility, productivity metrics provide information specific to agile processes, which companies can use to measure how quickly and effectively they respond to change.
While these numbers are important to know, executives should not focus solely on them. More important is knowing what these numbers say about the organization’s ability to adapt to change. There is a long list of agile metrics, but here are some of the more valuable for executives to know and understand:
- Sprint burndown reports the completion of work during a sprint. An agile team consistently meets its forecasted work for a sprint.
- Escaped defects are those that made it through delivery without being noticed. They indicate a lack of safety in the testing pipeline, says Ottinger. He notes that the count, trend and root-cause analysis for escaped defects are important for fixing the problem.
- Cycle time measures process capabilities and is directly connected to productivity. Cycle time measures the duration between when work begins on a request and when the item is ready for delivery.
- Lead time is the length of time from the moment an idea is received to when it is delivered to the customer.
Velocity: The Problematic Productivity Metric
Velocity gives the average speed of a team by measuring the average number of story points completed per sprint. It can be used to forecast when a feature will be released.
The challenge with using velocity to measure a team’s success is that velocity does not equal productivity, says agile coach and scrum master Tomas Kejzlar. Velocity does not provide value, and when velocity is used to measure performance, the company’s culture can become one of speed instead of value, which is antithetical to the foundation of agility.
Kejzler notes that focusing on velocity can often backfire because once people know they are being evaluated against a certain metric, they will do whatever it takes to improve or to achieve a goal, no matter how much damage they cause along the way. This is known as the Hawthorne effect.
Kejzler also suggests that Goodhart’s Law applies to measuring velocity. This law states that when a measure becomes a target, it is no longer a useful metric. A focus on velocity dilutes the information to be gained by knowing the measure.
Measure Metrics That Make Sense For Your Organization
The most important thing to remember when deciding which metrics to use is that the chosen metrics have to make sense for your organization. Amity University researchers Monika Agarwal and Rana Majumdar argue that optimal agile metrics should be simple, definable, objective, easily obtainable, valid and robust.
Agile metrics are an indicator for a pattern change, not a performance driver. Agile coach and scrum master Stefan Wolpers offers three rules of agile metrics that are helpful for leaders when trying to decide which metrics to track:
- Only track metrics that apply to the project team, not the individual.
- Avoid measuring parameters just because they are easy to follow. Metrics should be tailored to the organization.
- Record context with the data to help distinguish the solid data from the noise.
Chosen metrics are not set in stone, however. Be sure to recheck them frequently to make sure they are still relevant, says scrum trainer and agile coach Mark Levison. He suggests asking whether the metrics are still driving change with questions such as “How will this measure help our customer?” and “Does this measure have a ‘best before’ date?” Levison advises dropping metrics that are no longer helping.
When reviewing your metrics, remember that in agile it is not about the numbers but about delivering the best value to customers. Ottinger summarizes it perfectly: “We don’t want to make people good at measuring, we want to measure how we are making people awesome.”
Ensure Technology Solutions Support Agile Processes
Software solutions play a key role in a company’s ability to respond to change. The right technology can facilitate agility by making the authorization process much more efficient, says Artemis’ Chief Defense Industry Strategist Joe Kerins.
For example, Kerins explains, some companies can have as many as 200 changes in one month, which can be extremely costly in terms of time to completion. But with the right software, productivity measures such as cycle time and lead time can be greatly reduced thanks to automated approval processes, moving projects along more quickly in the face of change. With more time on their hands, team members can enjoy more slack time to focus on process improvements, another key tenet of agility.
Armed with a flexible software solution, program managers can maintain more project flexibility and encourage process improvements, helping to support an organization’s agile processes.