Interview: with Barry O’Reilly - outcomes over outputs

posted on

February 5, 2020

We interview Barry O’Reilly to learn why organisations should shift to Outcomes over Outputs as their way of working, and what benefits this can bring.

Managing Outcomes Over Outputs is more than just semantics.

One of the most consistent observations we have made here at Just3Things when working with our clients is the exponential difference in organizational culture, productivity and effectiveness when leadership and teams manage outcomes over outputs.

We spoke to one of the leading experts in this field, Barry O’Reilly. Barry is the author of recently released Unlearn: Let Go of Past Success to Achieve Extraordinary Results, and co-author of the international bestseller Lean Enterprise: How High Performance Organizations Innovate at Scale — included in the Eric Ries series, and a Harvard Business Review must read for CEOs and business leaders.

He works with many of the world’s leading companies, from disruptive startups to Fortune 500 behemoths, to break the vicious cycles that spiral businesses toward death by enabling culture of experimentation and learning to unlock the insights required for better decision making, higher performance and results.

J3T:

Hi Barry! Let’s start at the beginning – what is the difference between outcomes and outputs?

Barry:

It feels deceptively simple but can make a huge impact on how you, your teams and organisation perform:‍

Outputs are what you expect to ‘produce’, the products, goods and services which result from an intervention.
Outcomes are quantifiable changes in behaviour you will have caused due to the output you created.

J3T:

So why is it better for teams and organisations to define Outcomes over Outputs?

Barry:

‍An outcome is the level of performance or achievement that occurred because of the activity, product, good or service your organization provided. More specifically, the change in customer behaviour that occurred. Outcome measures are a better indicator of effectiveness than output measures. For example, if you completed the activity on time, on budget and in scope, but no customers use your product, would you consider the effort a success? Outcomes quantify and assess how the product or process you created enable customers to achieve the impact they are aiming for.

Spending the time to define outcomes over outputs is always trickier but gives lots of advantages, including:

Allowing teams to create a shared understanding of why they are working on an initiative and what success looks like.
Open up options for how teams could achieve the outcome they are aiming for.
Increase the meaningfulness of work because teams can tie their efforts to results and gauge if they moving towards the desired outcome they seek (or not).
Creating more of an experimental approach – defining customer outcomes and asking teams to take responsibility at this level opens up the opportunities to experiment with outputs. Did X move the dial? No? In which case let’s try Y.
Balancing the need for long term planning with a ‘test and learn’ approach. Most large enterprises are adopting agile or lean ways of working based on the promise of delivering better customer outcomes faster — yet the majority still measure output as success. It can be daunting for stakeholders who are used to annual planning cycles and predictive controls such as time, budget and scope to switch techniques to manage uncertainty. Defining outcomes help engage stakeholders in valuable dialogue on what behaviours they expect to see from customers as indicators of success over planning every feature that they will release that year with little real insight and limited customer feedback if they are useful, valuable or even needed. Ask stakeholders what they expect customers to be doing differently as a result of the output you create, then measuring and monitoring this can be a great way to help them get started.
Not creating a culture of ‘busy fools’ – in many output driven organisations we see lots of activity and people working really hard, but without stopping to ascertain whether what they are doing (output) is creating any value for the customer or organisation (outcome) until the output is full complete. Our mission should be to learn as quickly and cheaply as possible if our effort is driving the results we’re aiming for.

J3T:

Sounds great! So how do organisations start?

Barry:‍

I encourage teams to think big but start small, and learn fast as they explore how to deploy an outcome-based approach to their portfolio of work. Don’t change everything all at once — start small, tiny in fact. Identify one product, or new feature or backlog item, start there by trying the following:

Keep asking ‘why are we doing X’ – for example why are we releasing this feature? What difference will it make and to whom? What change in customer behaviour do we expect to see?
Don’t get hung up searching for the one perfect metric or baseline data — start where you are and define what customer behaviour you would expect or would be different if the output was available to the world.
Try to measure rates and ratios over totals — aiming for a percentage increase is better than saying ‘we can’t define this outcome and so we will see if we can get five customers use it’. Instead, say ‘60% of customers must use the feature to test its valuableness, and support an experimental approach’
Don’t forget that outputs can also be constraints to foster innovation – we typically think about outputs as a control metric (e.g. deliver this project with $1,000 or release this feature by this date) which force feedback cycles to happen, and reflect on our results. Use output metrics within reason as a forcing function for innovation, for example an outcome a customer success team might be responsible for could be increasing customer retention by 20% in the next month.

J3T:

What do you see as the key challenges to organisations shifting to outcomes over outputs?

Barry:

‍One of the biggest challenges is that people are so used to managing to deadlines, budget and scope that they don’t know how to measure value in any other meaningful way.

When I present the idea of looking at outcomes they can see the benefit but don’t know how to start writing outcome based measure of success — it is daunting.

Another barrier is trust between ‘business’ and technology stakeholders in organisations. Sometimes business partners can feel that you are trying to trick them, plus they feel comfortable managing outputs because it’s the way they have always managed initiatives.

When people are time poor, the pressure is high and they have been sold new process as the way to succeed, their appetite to experiment is much lower. This is why thinking big but starting small is key, and showing people how the method works could be a better approach. The breakthrough comes by working, learning and unlearning together. Review your existing initiative dashboards, notice if the measure is all output or outcome based. Try to identify and introduce one new outcome based measure on your dashboard that you can use as a pilot for a new way of working — then scale from there.

J3T:

What if the metric is a lagging indicator?

Barry:

When people start to embrace outcomes over outputs they often use metrics such as revenue, profitability and customer satisfaction as a measure of success — but these are all lagging indicators.

Businesses and teams need to think about how to also create leading indicators as the outcomes they are aiming for, and behaviour they know mean the customer is on the path of success.

For example, at Netflix increasing revenue through customer retention of the service is a key outcome for the business. But if you’re working there, waiting each month for customer retention results, and even correlating your work to such a top line metric such as customer retention, is challenging.

That’s why creating leading indicators for the outcomes you are aiming for is important and more actionable for the team. Therefore, if you’re working on the Search team at Netflix, and know retention is important, you can ask a question such as, “what customer behaviour in search would be a leading indicator for retention?” Maybe you’ll say the lead time from searching to starting a movie. You can then model and measure how quickly people can search for, discover and start a movie via your search functionality and see if that impact lagging indicators like retention rate of those customers.

This takes work and skill but ideally you would look at a mix of leading and lagging indicators— you need to be thoughtful and have checks and balances. But getting started is the first step, so start small and learn fast what works for your context.

J3T:

What other behaviours can stand in the way of teams focusing on outcomes?

Barry:

Leaders struggling to devolve decision making – or on the flip side, teams being afraid to make decisions.

The incentives in place in a business make a huge difference too – leaders can say that the team should experiment, try out their ideas to succeed and work together toward outcomes, but if leadership behaviour shifts when bonuses are on the line, or paid out to individuals on completion of projects on time, budget and scope, this is totally mixed messages.

For me, the trick is to find the cadence for your context; how do we get into the rhythm of making smaller more frequent bets and outcomes with leading indicators, rather than creating big upfront plans with the output predicted in terms of time, budget and scope ? Every company has different requirements for different levels of fidelity of plans, and some of this will also vary by stakeholder.

Figure out how best your teams can keep stakeholders updated with the bets that they have made and whether this has moved the dial on the outcome metric you’ve agreed together. This is a very different way of working than planning everything out at the beginning of the year, but the benefits you can realise in terms of organisational culture, productivity and effectiveness once you learn can be extraordinary.

J3T:

We hear a lot about psychological safety – how important is this?

Barry:

The quality of safety in organisations will directly impact the quality of information that you get.

Ensuring that individuals and teams know how to design safe-to-fail bets is important to enable and make sure they feel they can learn from mistakes. How openly and honestly teams can and are willing to share their learning, so that others can jump up the learning curve more quickly, is an outcome every organisation and leadership team should be aiming for.

You can think big, start small, and learn fast tomorrow by asking your teams: ‘on a scale of 1-10, how safe do you feel to share open and honest information?’ Accept the score they give, and start small to improve. Ask what you can do as a leader to help improve that score by half a point. Pick one suggestion for a week, try it and learn what works and what doesn’t. Role model your efforts to unlearn outdated behaviour and thinking as you seek to create a create of experimentation and learning to help others succeed.

‍