To calculate impact we need to focus on Outcomes

To create measurable, positive change, we need to focus more on Outcomes.

We hear a lot about impact investment but what exactly does “Impact” mean. Oxford Dictionary defines impact as: “A marked effect or influence, to have a strong effect on someone or something.” Unfortunately, many impact or mission-driven businesses and impact investors struggle to correctly report on that impact.

They overly emphasize their 'outputs' or "the amount of something produced" but not the 'outcomes' or "the way a thing turns out." AKA: The eventual impact / influence.

The customisation often required to define and subsequently measure outcomes limits the ability of impact investors and mission-driven organization leaders alike to assess the success of their organizations in regards to their overall mission and objectives.

To share insight on this problem we spoke with Pyarali Jamal.  Pyarali has worked in the impact and social enterprise space for over 20 years across North America, Europe and Central Asia, including serving on the boards of UK impact investment organisations. He also has experience in the private sector spending significant time in banking, at EY and as a strategic consultant for C-suite level executives. Seeing the issues and challenges around effective reporting, Pyarali acquired training in social impact monitoring and evaluation in 2012, and began to run educational workshops on “Outcomes-based Strategic Planning” for boards and senior management.

Pyarali discussed with the Nossa Capital team his views on how impact investors and impact organization leaders can improve their measurement and reporting of outcomes. He emphasizes that in order to maximize impact, leaders and funders alike need to be Outcomes driven, not Outputs driven.

The discussion is summarized below.

Even though there is now a greater emphasis on "outcomes", we do not see enough reporting of outcomes in impact investing activities.

Pyarali began by sharing:

"I've seen firsthand that we don't effectively report outcomes. This comes down to the underlying issue that people in this space don't have a good enough understanding of the difference between an outcome and an output.”

It is common to see reports on outputs he shared, however, outputs do not effectively tell us if we have had an impact on the issue we’re tackling.

Scenario: You’re trying to improve unemployment rates in a certain region in East London. To do this you provide skills training around interviews, CV writing, business education etc.

If you’re measuring how many people went through that training that would be an output.

The outcome would be how many of them actually found [better quality] work afterwards within a reasonable period of time (eg: 3 months). Another outcome would be how many of them maintained that job, or moved onto better employment a year later. If the objective of the programme is to reduce youth unemployment, then these are the types of outcomes the organisation should measure.  

Organizations must stay focused on how they can better achieve their outcomes. From an outcome perspective, considering the scenario above, it could be that the training wasn’t enough.

Focusing on outcomes could show an organization that they need to do more than training. It could be that they will have a greater positive effect by investing additional time and resources to create a pathway that leads to new job opportunities.

What are the key questions to ask when considering outcomes versus outputs?

The first step is making sure that the problem actually exists. You cannot pick what to measure if the problem isn't there, or if it isn't clearly defined. Questions you can ask yourself can include:

Who is it you're trying to assist?

What is their need?

Is this a priority need or a serious issue they face?

How do you know this problem really exists?

And, importantly, what will you measure to determine if your programme / initiative has directly contributed to meeting that need or addressing that issue?

In the case of unemployment in a certain neighbourhood, these questions would be:

How do you know that unemployment exists in that neighbourhood?

Where is the evidence for that?

It doesn’t need a lot of steps to justify a problem is really there. Sometimes it could be simply good and reliable anecdotal evidence but... "if you have the statistics it is even better."

It's essential to start with a clear understanding of the problem because, if we are not confident that a problem or issue really exists, we could be looking for a solution for a problem that is not there. Comparing this to a traditional for-profit business, the question would be: "How do you know there is demand for this product or service?"

Coming back to the scenario above, if an organization is focused on improving employment rates was looking for funding, the key question for that type of organization to answer would be:

“How do you know if you're training thus far has been useful and has actually led to job opportunities?”

Many organizations get stumped at this point. They haven't thought this outcome through.

What can the impact sector learn from the private sector about outcomes reporting?

Demand analysis:

How do we know a problem exists?

How do we know that there's demand for this product or service that you will launch?

Is the initiative or programme financially and operationally viable?

Even in the private sector, many start-ups and early stage companies struggle to clearly answer similar questions to validate demand for the product or service. As a result, a lot of capital that gets put into companies that don't go anywhere, or they're unable to raise sufficient capital. In order to secure additional capital in the future as well as deliver on outcomes, the social sector needs to think more commercially.

Sales and customer service:

How are social sector ‘products’ effectively brought to market?

What are their economics? Is it commercially sustainable?

Is the company marketing and servicing its customers in an effective way?

You run sessions with impact organizations on outcome reporting. Could you share a bit more about how that process works?

The sessions are targeted to decision-makers at the board and senior management level. I've done this for not-for profit and for-profit businesses alike.

I customise each session, but they all have certain common elements. For example, I spend the time clarifying key terms. Just as traditional for-profit companies need to clearly understand the difference between gross profit, operating profit and net profit, mission driven organisations must have a clear understanding of the difference between objectives, outputs, outcomes, impact, and KPIs. That may sound obvious, but in my experience, it isn't obvious to many decision makers, whether they be impact investors or investees. The difference in these terms is as different as understanding gross margin vs operating margin vs net profit margin.

Output vs Outcome vs Objective

Training exercise: Create a list of example metrics for your organization and determine if they are outcomes or outputs.  And make your outcomes "SMART".

My sessions also include practical examples, including examples customised for the organisation, of outputs and outcomes. This is when I engage the group, often in breakout groups, and ask them to determine if the example is an output or an outcome, any why. This is an effective way to practically apply the framework. Below is a diagram which shows the overall Outcomes-Based Strategic Planning Framework.

A key takeaway is: Monitoring and evaluation cannot be an afterthought.

Just like we need accounting systems to measure revenues, gross margins, operating margins and profit. As you define your “target outcomes” you also need to determine how you are going to monitor and evaluate them. This could be done via surveying, observations, focus groups, one on one follow-ups, etc.

SMART Outcomes stand for:

S = Specific, M = Measurable, A = Attainable, R = Relevant, T = Time-based

Going back to consider the outcome measurement from our previous scenario:

How many people found work within a reasonable period of time (eg: 3 months.)

The output was: “The number of people who successfully went through a training program.” Another related / supporting output is, "The percentage of people who found the training highly informative and practical." But, it's important to note that one can find something "informative", but not necessarily "useful".

Examples of suitable "SMART" outcomes would include:

Did the people who went through the training find jobs in three months? (i.e. was it "useful")

Did they retain employment a year later or move into a better job?

The diagram below can help you visualize the interdependencies of:

Mission -> Objective -> Outcome -> Output.

Have more questions about how to define outputs versus outcomes as an Impact investor/organization? You can contact us at Nossa Capital or reach out directly to Pyarali on LinkedIn.

‍