Putting the ‘ROI’ back into SROI
How to use Social Return on Investment (SROI) to compare social investments by SVA Consulting’s Executive Director, Duncan Peppercorn.
The majority of Australian social investors (including government departments) and the organisations that they fund and support have now heard of SROI. You may be, or work for, one of these investors and already know that it is a tool to evaluate the efficiency of a program in using your money to achieve social ends.
You probably also know that SROI is a form of cost/benefit analysis in which social outcomes are assessed and then have a financial value associated with them. The total financial value of the social outcomes achieved is compared to the investment required to achieve them (the benefit is compared to the cost), and a ratio, called the SROI ratio, is produced. If this ratio is greater than one (i.e. the benefit is greater than the cost) then the investment can be seen as efficient and wise.
You may also believe that SROI allows social investments to be compared, or that a ratio of 12:1 is objectively ‘better’ than a ratio of 6:1, or that there is an ‘acceptable’ or ‘satisfactory’ level for the ratio. But if you have spoken to an SROI practitioner, or investigated SROI through the international SROI Network, you will have been rapidly dissuaded of these notions. You will have heard that an SROI analysis is less about the ratio and more about the process; what it uncovers about how the program works and the impact of the program.
In fact, you will have learned that the great strength of SROI (compared to other social cost/benefit or evaluation tools) is the agreed principles underpinning the analysis[i] and the rigour with which SROI practitioners are vetted to ensure accreditation.
None the less, that ratio won’t go away! A Return on Investment (ROI) is an objective measure that allows commercial investments to be compared, isn’t it? So, surely, an SROI does the same for social investments? Despite all our efforts to head this conversation off, almost every client for whom we have completed an SROI in the last 10 years (and there are more than 70 of them) has asked “is this a good ratio? How do we compare to others?”.
The reasons that you should not use SROI to compare programs are these.
Firstly, and most importantly, you cannot objectively compare curing a sick child to getting an Indigenous kid through high school. Objectively, one is not ‘better’ than the other, regardless of the apparent SROI ratio. At a more granular level, even comparing a program getting Indigenous kids through high school in the city with one in the country is hard. While the latter is likely to have higher costs and a lower return, that doesn’t make it a ‘worse’ investment if your interest is in helping kids in the bush.
Not all stakeholders value a particular outcome in the same way. Similarly, not all stakeholders value the same outcomes.
Secondly, anyone who has completed an SROI analysis will understand that, while we have techniques to place dollar values on social outcomes, these are only real cash savings in those cases where the outcome actually has a dollar impact, such as when an unemployment benefit is saved. Otherwise, the financial valuation of outcomes is subjective – we can use sophisticated approaches such as choice modelling or revealed preferences – but the person getting the benefit may place a different financial value on an outcome to that placed by, for example, their family.
Even when the outcome has an objectively correct dollar impact, that benefit may not actually accrue directly as cash to the investor themselves, but to someone else. For example, a philanthropist investing in a project to reduce criminals reoffending may save the government incarceration, legal and policing costs. It is only if the government made the investment that the investor would actually receive the financial benefit, and even then it might accrue to a different department and budget.
Valuation reflects preferences
Any ROI is actually a measure determined by the investors’ goals. Even commercial investors do not simply go for the highest ROI. They may have different appetites for risk; they may want returns sooner or they may only be interested in those sectors that they understand and are excited about. All of these considerations may be used with, or without, an ROI to make a commercial investment decision. There’s more to the effective use of ROI than a single, simple number, and the same is true of SROI.
When we decide on the proxies to use in a valuation, we are making a judgement. Not all stakeholders value a particular outcome in the same way. Similarly, not all stakeholders value the same outcomes.
Elsewhere in this issue of the SVA Quarterly we have reported on the work done to develop an innovative and simple approach to measuring the social impact of the IBA business.
In this work we reached agreement with the investor – IBA – on both the outcomes that it wished to invest in and to measure, and on how it wished to place dollar values on the social benefits accruing from its investments. We took into account the perspectives of the beneficiaries, but the investor was the final arbiter.
As a result, the same outcomes, financial proxies and deadweight calculations are being used across the IBA Equity and Investments Program portfolio. In this case, ratios can therefore be compared (although IBA does not currently use this to ‘compare’ investments since it is primarily interested in improving performance). New investments can be valued, targets set and performance against existing investments compared.
MySROI: a new way to use SROI as a comparative measure
Building on the IBA experience, we can now think about SROI in a different way. We must continue to stick rigorously to the principles of SROI and to the analytical process. But in placing a dollar value on outcomes and creating a SROI ratio, we can be guided by the investor in those situations when the investor has requested the SROI analysis.
If we are working for the program owner rather than an investor, we can provide them with a comprehensive set of options and the required understanding to allow them to present their SROI to an investor with a much more sophisticated message about the ratio:
“We understand what outcomes the program has, and to what extent we are achieving them.”
“OUR strategy identifies which outcomes WE seek to achieve, and WE have financially valued these outcomes in a way that is appropriate for us. And we believe that investors in us and our program are getting a return of…”
“However, YOU can choose which of them you value. Here are a variety of ways to place a financial value on those outcomes. YOU can choose which you wish to use, or whether to take a different approach entirely. You can establish your own return ratio.”
For example, imagine a program similar to some that SVA Consulting has supported, which works with Indigenous kids to keep them in school beyond year 10. The program might have the primary objective to increase school retention, but might also increase self-esteem, reduce substance abuse and anti-social behaviour (including theft), and improve family cohesion.
The SROI process would uncover all of this. It would understand the benefits to all the stakeholders: the kids, the schools, the community, the police, families, etc. It would comprehensively identify and measure the outcomes achieved by the program. If the analysis is done correctly, everyone should agree that these outcomes happen for each stakeholder. This is the bulk of the work in an SROI analysis.
An investor (including government) can then choose the valuation approach that reflects their own perspectives and preferences.
However, beyond this we must recognise that decisions become subjective. The dollar value placed upon ‘less theft’ by the police may well be different to the value placed by the young person, their parents or even the community. Indeed the young person might value completion of school much more (or less) highly than a notionally objective financial value (such as the expected lift in whole-of-life earnings).
It is appropriate to identify options for the dollar value placed on the outcomes and present these transparently in the resulting report (‘transparency’ is one of the principles of SROI).
An investor (including government) can then choose the valuation approach that reflects their own perspectives and preferences. So long as they are consistent in their approach between different programs, they can then use SROI to compare.
By adopting this change to how we present (and calculate) the contentious ‘SROI ratio’, SROI will be much more useful to investors in making wise decisions. So, government department A might take the view that while it is happy to improve family cohesion, etc., its primary objective is school retention, and that the appropriate valuation of school retention is the lift in whole-of-life earnings. In considering an investment in the program it would look only at this outcome and this financial valuation. Similarly it would use an identical analysis to compare investments.
By adopting this change to how we present (and calculate) the contentious ‘SROI ratio’, SROI will be much more useful to investors in making wise decisions…
Private philanthropist B might take the view that school retention is entirely the responsibility of government, that they are interested only in reducing substance abuse, and that they value the latter by the self-reported increase in quality of life for kids who are drinking, smoking or otherwise abusing drugs less. Philanthropist B’s SROI ratio for the program will be different from department A’s. That doesn’t change the achievements of the program but reflects that different investors value those achievements differently.
In practice this approach adds some work – particularly for investors who have to decide what they value and how they value it – but it pays massive dividends in creating a truly comparable measure of the performance of social investments; one that can be used by an investor to compare programs and decide where to invest.
For the SROI practitioner, this approach is consistent with the principles of SROI in that it increases the involvement of a key stakeholder: the investor.
Continuing to develop SROI in Australia
Social Ventures Australia was at the forefront of the introduction of SROI in Australia, eight years ago. We firmly believe that the methodology illuminates how programs work, builds a framework for measurement and evaluation, and helps management to improve social outcomes.
Changing the way that we actually do SROI analyses is necessary if the methodology is going to develop and improve. (Jeremy Nicholls, CEO of the international SROI Network, compares SROI today to accounting 30 years after the first profit and loss statements were drawn up – in Sumeria, a thousand years before the Christian era.)
By adopting this change to how we present (and calculate) the contentious ‘SROI ratio’, SROI will be much more useful to investors in making wise decisions about the programs and ventures that they choose to support, and we can then put the ‘ROI’ back into SROI.
[i] The principles of SROI, as agreed by the International SROI Network, are 1. Involve stakeholders; 2. Understand what changes; 3. Value the things that matter; 4. Only include what is material; 5. Do not over claim; 6. Be transparent; 7. Verify results.
Duncan Peppercorn will be speaking at the conference: Measuring Social Outcomes in Melbourne on 15-16 October 2013.