Why productivity shouldn't be the key measure of agile transformation

By asking teams to assess how aligned they are to the agile principles, we unlocked their ability to improve and grow.
Register or Login to like
Consumer Financial Protection Bureau on open source and "growing the pie"


As a principal agile coach, my focus is to energize and inspire developers to adopt and follow through on the agile frameworks across our teams. Initially, I failed, as I didn't really understand the mindset of teams that have been working in an open source ecosystem. My assumption was that since agile and open source values align so closely, the barrier for adopting agile frameworks would be low and our people would take to it like fish to water.

In the beginning, it was easy. There was very little resistance, except from a few who had really bad experiences in the past or believed that agile is not cut out for open source development efforts. Since most of the team members were on board, we went ahead and adopted scrum for most of our teams.

Three months into our agile adoption, we noticed that a few teams were performing well while others were struggling, so we felt it was time to start collecting metrics to objectively measure performance and to identify issues that led to success or struggle. We actively worked with two of our teams to start assigning story points to user stories so that we could determine each team's velocity. It worked well, and, working with the teams, we were able to identify barriers and start fixing them.

Short-lived success

In my enthusiasm (which didn't last long), I sent out an email to all my teams asking them to follow the same process. I wanted each team to start assigning story points to user stories so that we could measure their performance and take corrective actions at the end of the sprint.

What followed was a vociferous barrage of email replies from some of the best and brightest people about why this was not a good idea and why it would end badly. Responding by instinct, I started replying to the thread, arguing why this will help us and giving various analogies. The more I tried to defend my plan, the more resistance I encountered.

Pretty soon I gave up, as winning this battle by enforcing process would only make things worse, and I realized I would lose all the goodwill that I initially gained. I thought I'd pick it up later when the team grew comfortable and gained maturity with respect to agile processes and techniques.

My mentors, long-time Red Hatters, had been observing this unfold from the background. They helped me look at it from a different angle. The leaders in our organization were interested in finding the positive impact of adopting agile frameworks in a holistic sense; it wasn't just about the performance of the teams. They wanted to know things like:

  • How agile is our organization?
  • In which areas do we need to improve?
  • Is there a way to visualize progress?
  • What metrics are being used to track our agile transformation?

Unfortunately, most of the available tools and techniques track the productivity of teams or how well the processes are implemented, rather than getting into the heart of the problem.

Instead of relying on velocity or adherence to processes, we measured the teams' true agility by assessing their growth and capability, not just productivity. We felt this could be done by reflecting how closely their behaviors, actions, and decisions aligned to the core agile principles.

We introduced a radically new approach to tracking our journey of agile transformation. We started by creating an environment where these four behaviors were actively encouraged across the teams in the context of their agile journey:

  • Experiment: Let us keep it fresh
  • Share: Let us start with what we know
  • Listen: Let us observe and reflect
  • Learn: Let us develop ideas and understanding

When teams were given the freedom to experiment and develop their own flavor of agile, there was a lot less resistance and a lot more enthusiasm for continuous improvement. Since the teams were actively sharing, we started cross-pollinating the ideas and information into other teams. The teams were now mature, and self-correction started to happen whenever something didn't work out as expected.

A new approach: Agile Reflection Deck

We developed the Agile Reflection Deck, where we divided the 12 agile principles into three templates consisting of four principles each. After each iteration, the team would reflect on how aligned they were with each of the principles and rate themselves on a scale of 1 to 10.

As the 12 agile principles are universal and generic, the teams had to engage in a lively discussion to gain a consensus on what each principle meant for their team. Then they had to average the team members' ratings and plot that number on the axis where the agile principle was listed.

Agile Reflection Deck

As we were a highly distributed team, we used online survey tools to capture the ratings during video conference sessions and then displayed them on the Agile Reflection Deck. Once we had the four ratings mapped onto each axis, we connected the dots to form a quadrilateral. Over time, the shape and size of this quadrilateral gave us a clear picture of the teams' agile mindsets.

By allowing the teams to self-reflect and analyze how closely they were aligned to the agile principles, we were able to unlock the potential and ability for the teams to improve and grow with each iteration or checkpoint. We had truly empowered the teams to map their journeys and make self-corrections to chart their course to agility.

Ranjith Varakantam
I am part of the amazing Red Hat Developers Program and lead the Agile Transformation as the Principal Agile Coach. Our goal is to provide tools that make it easier for developers to succeed in their everyday endeavours, right from planning to production.


I wish the story included more details about how we can know that the Reflection Deck helped. Was performance measured any other way, so as to correlate changing quadrilateral shapes to objective progress?

Thank your for your interest, I’ll elaborate on how it helped and then get into the second part of the question. By adopting this philosophy, we have demonstrated inherent trust in the team’s ability to assimilate and undertake the Agile transformation onto their shoulders. This created a better environment to have open conversations, which was previously tough when based solely or velocity of the team. This is because the nature of the projects that we undertake is quite complex and working with an open source environment adds to it. As we depend in other projects and communities to progress in our efforts which we have little or no control over.

Coming to the second question, I personally believe that once a teams have embraced the Agile mindset they are become much more capable of producing high quality work products. So for us, we realised measuring just productivity is of little value, if output is not properly aligned with business goals or expectations of the stakeholders. We rather want to focus on building the team’s capability to be able to perform in a fast changing environment where priorities could rapidly change in a relatively short period to time.

For example one of the Agile principle that revolved around stakeholders satisfaction triggered an energetic and enthusiastic discussion to identify the stakeholders. Needless to say the team quickly went beyond the product manger, business units, organisation and thought about community and end users. As you can imagine, when each developer in the team becomes capable of stretching their thoughts to understand how their piece fits into the big picture the quality of software they produce is much higher and also tightly aligned with the overall vision.

The artefact that is generated at the end of these discussions, helps us to visualise what the teams have identified and how they progressed over few months or weeks, before we revisit the Agile principle. It is like a footprint on our journey towards becoming truly Agile. For measuring performance, we observe and inspect is the quality of the deliverables at the end of each iteration and the impact it creates in the market and community.

For example when we release a new version of the product, we see the reaction and emotion it generates, either through comments or questions on blog posts, reporting of issues or number of downloads. As these are self-evident and tie back directly to the business drivers, it gives us a much more wholistic picture of how are teams are doing.

In reply to by Anon Y. Mous (not verified)

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.