The saying “If you can’t measure, you can’t manage” also applies to agile transformations.
When Julia stood in front of the management committee, she struggled to demonstrate the progress of the agile transformation. Confused by the fuzzy top-level indicators and unimpressed by the anecdotical operational metrics, the committee directed its attention to the one thing it understood well: the cost.
While it’s easy to measure something, measuring the right things is another story altogether.
If you don’t have the right performance metrics, you cannot demonstrate progress, invest your efforts where it matters, or even engage people, which might very well spell the end of your agile transformation. After all, with everything we’ve got to do, why would we spend money and energy on something we don’t even know the status of?
Finding the right performance indicators for an agile transformation is elusive. While it’s easy to measure something, measuring the right things is another story altogether.
Measuring too little
In one instance, the transformation team focused almost exclusively on the speed of delivery. And of course, it generated a host of negative side-effects—such as a decrease in quality—which the team was blind to because they were not measured.
Measuring too much
I still remember the gigantic dashboard where each of the twenty or so managers had his or her indicators arranged in a three-level hierarchy. The performance management model was a mathematical marvel. Too bad it was impossible to use; no one could keep track of the cause and effect between actions and metrics. It was particularly good at giving the impression of control, tough. The leaders pulled the plug when they realized it did not represent reality and it took each manager four hours a week to keep up to date.
Measuring the wrong things
One transformation team chose metrics that were comparatively easy to measure, like the maturity of agile teams based on how well they applied agile methods. Do they have a Scrum Master? Check. Are all their user stories in the tool? Check. And oh! surprise, while agile maturity increased, business-level metrics remained unchanged.
Four top-level agile transformation performance metrics
I’m going to share with you four top-level performance metrics and a few practices that help manage the performance of real agile transformations in complex organizations.
To identify our top-level metrics, let’s start from the desired outcomes.
To identify our top-level metrics, let’s start from the desired outcomes. What are we trying to achieve? How does it enable our vision? You’ll usually end up with variations of the following four agile transformation performance metrics.
Customer satisfaction
That’s a no-brainer. “Customer” means anyone who benefits from your products and services. For a commercial company, it’s the customers. For a hospital, it’s the patients.
A concern that often arises when measuring customer satisfaction in an agile transformation is that agility is certainly not the only driver for customer satisfaction. This is entirely true and it highlights the importance of integrating the performance management of the agile transformation with other performance indicators. Not connecting the agile transformation to a type of customer satisfaction metric would be a big mistake.
Speed of value delivery
Here we measure the frequency of delivery of business value. Not technical delivery, nor delivery to an internal stakeholder, but delivery of stuff that benefits the customer.
This performance indicator is strongly related with customer satisfaction but also complementary. You will not satisfy your customers for long if you can’t keep up with changes. On the other hand, delivering fast low-quality products, for example, will ultimately hurt your customer satisfaction.
Learning & Innovation
Innovating—whether in products or ways of working—is one of the reasons why organizations want to become more agile. Learning and innovating are necessary to keep your products and services competitive and to attract and retain talent.
The key here is to measure the knowledge produced through experiments.
The key here is to measure the knowledge produced through experiments rather than the number of innovations. Indeed, innovative organizations see innovations as the eventual and unpredictable results of “trying stuff” and systematically learning from experiments. Experiments succeed if they answer the questions and produce knowledge, even if the answer is “it doesn’t work.” It's quite logical, really: if you try something that has never been tried, then you must be allowed to “fail”; and to get better, learnings must be systematically collected and reused.
Therefore, the number of experiments performed is a good metric for learning and innovation.
Workers’ happiness
Note that I use here the term 'worker' and not 'employee'. In today’s trend toward an ever more flexible workforce, it’s important to reconsider the boundaries of the organization. If you collaborate with partner organizations, consultants, interns, freelancers, and other people who are not on your payroll, take them into account for this performance metric, because your success depends as much on them as on your employees.
Organizations increasingly state talent attraction, retention, and engagement as explicit goals for adopting agile. It is no surprise that agility has become a powerful instrument for these purposes since it encompasses values such as collective intelligence, autonomy, respect, and transparency, among others.
Organizations increasingly state talent attraction, retention, and engagement as explicit goals for adopting agile.
It is entirely possible to perform well according to the three indicators above and still make people miserable. Although some organizations manage to perform business-wise while treating their workers poorly, as they churn through employees, disengagement will eventually catch up with them.
So, at the risk of saying the obvious, caring about people is not just a good human trait, it is also paramount for sustainable organizational performance.
Good practices (and pitfalls to avoid)
Leading indicators
The four agile transformation performance metrics proposed above are lagging metrics, which means that there is a significant delay between the actions you might take and their effect on these indicators. For example, if you tweak your product development process to deliver products updates more often, due to training, process changes and other adjustments, it might take several months for the changes to finally accelerate the speed of value delivery.
Additionally, our four top-level indicators depend on several types of actions, which might make cause-and-effect not so obvious. For example, if you notice a jump in workers happiness, is it due to the new multidisciplinary teams or rather to the upgraded holiday policy?
For these two reasons, top-level indicators are not very actionable as such. To act on these indicators, we need observable signs and behaviors that are strongly correlated with them. For example, for customer satisfaction, the degree to which product managers relate to customers might provide a basis for a good leading indicator.
For each of our four top-level performance indicators, identify a few leading indicators that make sense in your organization.
Co-create the agile transformation performance metrics
Who defines and updates the metrics is as important as the metrics themselves.
Run co-creation activities to elicit and periodically update the metrics. Similarly, include stakeholders from a variety of areas for retroaction and adaptation. Managing the performance of an agile transformation is an agile process.
Avoid, for example, to rely on a governance committee to manage performance metrics in isolation. People might feel alienated and not play along, even if the metrics are the right ones.
Use strong visuals
Your key metrics should be summarized in a simple visual, legible on a letter-sized page or a laptop screen. In one instance, we had four quadrants (one for each top-level metric), each with 2 or 3 leading indicators. Each indicator was depicted as a simple gauge. You could know what was going on at a glance and act accordingly.
Aim for improvements rather than specific targets
Instead of setting a specific target, for example a monthly frequency for the speed of value delivery, focus on trends. Is it getting better or not? In this regard, the use of objectives and key results (OKR) is often more effective than traditional KPI’s.
Leverage transversal metrics
Transformation teams often make the mistake of defining performance metric in a way that each metric is owned by one and only one manager. Indeed, managers typically agree to be responsible only for performance metrics that are fully under their control. Since most meaningful metrics depends on collaboration over multiple functions, it puts pressure to “silotize” the metrics.
However, the assumption that having a single owner for each metric would yield higher overall performance, is flawed. On the contrary, having transversal metrics depending on multiple managers and functions fosters collaboration and generates higher overall performance.
Don’t be afraid of subjective metrics
The word “subjective” often has negative connotation, which is why many avoid subjective metrics—metrics subject to interpretation and not calculated based on factual data. Contrarily to this belief, subjective metrics are perfectly fine, on condition that you have the right subject persons.
Subjective metrics are perfectly fine, on condition that you have the right subject persons.
For example, surveying workers to rate their happiness is fine because workers are the subjects. Whereas surveying managers to rate workers happiness is not (or at least not alone), because managers are not the right subjects for this metric.
Subjective metrics also have the added benefit to engage the people being asked.
Beware of vanity metrics
There is a lot of pressure to show that you deliver results. Agile transformations are no different. It’s very tempting to unconsciously use a metric that makes you look good.
In one case, the agile transformation team concentrated on producing tools, for example descriptions for agile practices and templates. The number of tools produced quickly became an object of pride and a key metric, even though most of these tools were not actually used or did not satisfy their users. In another case, the number of people attending agile training was put forward, but whether they put their learnings in practice was not taken into account.
Vanity metrics are often metrics that you have direct control over and free you from the need to assess their real benefits. Properly linked to top-level performance indicators, these are legitimate operational metrics, but abusing them is a sign that the agile transformation is focusing too much on the means and not enough on the outcomes.
If Julia had applied the four top-level metrics and followed the good practices proposed above, her meeting with the management committee would have gone very differently. The executives would have understood how the agile transformation furthered their business goals and they would have been confident that the metrics represented reality.
Even better, agile transformation performance metrics can drive all sorts of positive behaviors and organizational changes. For example, the need to get together to improve customer satisfaction and speed of value delivery encourages the organization to redesign itself along value streams. Similarly, there will be a welcomed pressure to simplify bureaucracy and reposition decision-making. Likewise, workers’ happiness helps improve psychological safety and workers flexibility, which in turn improve productivity and innovation.
Done right, managing the performance of your agile transformation will generate a virtuous circle.
Watch webinar video: Strong Metrics to Show that Your Agile Transformation Matters
Related content:
Bruno Collet advises and coaches leaders to develop the individual and organizational capability to anticipate and react to changes better and faster. In one word: Agility. His approach relies on action-learning with concrete practices, skills and behaviors. Bruno Collet executed top-level missions with several organizations internationally. He is also a recognized speaker, author and is accredited ICAgile instructor. Bruno Collet holds an MBA, MSc., as well as several certifications such as PMP, PMI-ACP, ICP-LEA (Leading with Agility), ICP-ORG (Adaptive Organization), ICP-ENT (Enterprise Coaching) and ICP-AHR (Agile Human Resources).