MAILBAG: How to Define Success for Your Agile Initiative

From the mailbag:

Dear @TheBobWinter,

Our IT department is shifting from the traditional Waterfall approach to Agile. This will not only impact IT employees, but anyone within the company who works with IT on a project. We have a training plan, and employees will be educated via workshops on how to work in an Agile environment. Now, our  IT leadership wants to know:

·        How will we judge the effectiveness of the training?

·        How will the shift to Agile impact the business?

I would look at for the answer in your book, but I am on the road this week.

Collegially,

Scott in Scottsdale

+++

Scott in Scottsdale,

Nice to hear from you! The answer to your questions could and does fill countless books and articles – including Chapter 6 of my bestseller Agile Performance Improvement.

It sounds like you are cooking up a measurement plan to accompany your learning strategy. Here are some starter thoughts. Proving the efficacy of an agile program is tricky, because you can’t necessarily make direct connections between the training or the PMO efforts and the business results on the other end. However, you can demonstrate some good value in these three categories.

Practices
If you look up “agile metrics,” there are all sorts of ideas in the general category of ‘conformity to practices’. So, are the teams doing x practice (stand ups, planning, retrospectives, etc.)?  At some point management will probably want to know if teams are actually “practicing agile,” so they need to decide what that threshold is in terms of, say, “%age of adherence to practice.” Once your teams reach higher levels of maturity, these numbers will become less meaningful, since teams will learn that not all practices work for everyone.

Business people know that these agile practice indicators don’t pay the rent, but they typically do want to be able to say, “our people are doing agile.”

Team Performance
More importantly, the business will care about whether key indicators of team performance have improved in a measurable way. Typical metrics would be things related to speed (e.g., velocity or delivery rate) or quality (defect rate) or customer satisfaction (NPS).

The business should articulate what they are hoping to improve before kicking off the agile initiative, and a critical few metrics should be tracked. The underlying assumption is that whatever the focus on team performance, it has strategic alignment with what the business is trying to do overall.

Employee Engagement
Some executives consider it “fluffy,” but employee sentiment can be a powerful indicator of how things are going. Set a baseline and monitor employee engagement, both in general and with the Agile initiative. Research shows that engagement (or satisfaction) is positively correlated with discretionary effort and individual performance.

+++

Within each of these categories, there are countless possible measures. I recommend choosing a small number that will resonate with the key stakeholders. As such, any type of plan of this nature should be codified only after consultation with the business people. After all, they are the key stakeholders, and they should be on the same page with the desired outcomes.

The whole idea is: Are we getting better? How will we know that we’re getting better? Do the indicators represent what is important to the business?

I hope this helps. Let me know how it goes, and keep in touch.

Yours,

@TheBobWinter