The SVA Fundamentals for Impact enable organisations to assess whether they are being effective and how they can do better. International social impact consultant David Pritchard explains why and how the framework was developed.

Social sector organisations face a number of ongoing challenges. One of the most problematic, that gets to the very heart of their existence, is trying to identify if they are actually doing good, and how to do better. Over its 15 year history, SVA has wrestled with this effectiveness challenge, both for itself and for the organisations it supports. Recently SVA put resources into developing a framework, known as the SVA Fundamentals for Impact, to help address this.

The effectiveness challenge

A crude comparison is that social sector organisations strive for the social good while the private sector pursues private financial gain. In reality, the purposes of organisations and motivations of people within them are more complex: social sector organisations and their staff sometimes pursue their own ends to the detriment of others, while many companies and their employees are inspired as much by providing a valuable service to their customers as pursuing their own interest.

But while crude, this distinction flags a critical challenge for the social sector. Companies can assess and increase their effectiveness by focusing on a simple measure: profit. The better they serve their customers and clients, and do so efficiently, the more profitable they are. They can decide whether to invest in a new product or service, or try a new way of doing things, based on whether it is likely to make them more profitable or not.

How can social sector organisations assess their effectiveness? How can they decide what to invest in?

By contrast, social sector organisations do not have a simple measure of their effectiveness. They have to come up with their own way to define and measure their contribution to the social good. They may believe they are effective, but find it hard to know for sure. Moreover, they don’t have a simple basis for judging whether a new program or a change in their operations will make them more, or less, effective.

SVA has been addressing this challenge for many years. It has identified, applied, and advocated a number of methods designed to help social sector organisations in Australia assess their effectiveness and become more effective. Such methods include use of theories of change, strategic planning, outcomes management, and Social Return on Investment (SROI). But SVA has found these methods are not enough to allow it to fully assess and improve its own effectiveness.

SVA is an intermediary. Its contribution to the social good depends on how much it helps other organisations become more effective. The methods noted above are part of the puzzle. It helps social sector organisations organise their resources to achieve their goals and assess that achievement. But there is a piece missing. What if your goal is the effectiveness of others? There is no easy way of assessing whether SVA has achieved that.

How can social sector organisations assess their effectiveness? How can they decide what to invest in? How can SVA assess its own effectiveness and contribution to the effectiveness of others? These are the questions that inspired SVA to look for the organisational characteristics that are fundamental to having a positive impact. Are there such characteristics, and if so, can they be defined in a way to both assess effectiveness and guide decisions on how to increase effectiveness?

Looking for the ‘Fundamentals for Impact’

SVA is not the first to ask these questions. For decades, academics and practitioners have studied the problem of defining organisational effectiveness for social sector organisations in a way that is operationally useful. Indeed, there are three sources of knowledge to draw from:

1. Theoretical: what can theory tell us about what it is for a social sector organisation to be effective?
2. Empirical: what does research and data tell us about which organisational characteristics are related to effectiveness in practice?
3. Experiential: what do practitioners find matters in making organisations effective?

Theoretical

The common understanding of effectiveness is the extent to which an organisation achieves its goals and objectives. The closer it comes to achieving them, the more effective it is. (Though this is too simple as it implies that an organisation that has ambitious and difficult goals that remain out of reach is automatically less effective than an organisation that has goals that are easily met.) Academics call this the the goal attainment theory.

However, not only is it difficult to assess whether goals and objectives are being met, in practice this is not the only factor that leaders of organisations and their stakeholders focus on. This has led academics to create three other theories about what makes an organisation effective:

• It is able to acquire and maintain resources critical for its survival – the resource-dependence theory
• It is able to transform inputs into outputs efficiently – the internal congruence theory
• It is able to maintain stable relationships with its environment and meet stakeholder demands – the strategic constituency theory.^[1]

One point at risk of being overlooked with all four theories is that social sector organisations aim to promote the social good. For SVA this is reducing disadvantage in Australia. The focus of all four theories on the effectiveness of the organisation could exacerbate the problem of pursuing goals that may be good for the organisation, but may not help its ultimate cause – known as sub-optimising. Paraphrasing the great management theorist W. Edwards Deming, sub-optimisation is when every social sector organisation works for itself; optimisation is when every organisation works to help the cause.

So from a theoretical perspective, the SVA Fundamentals for Impact need to:

• Enhance the ability of a social sector organisation to meet its mission and goals
• Reflect the organisation’s ability to acquire and maintain resources critical for its survival
• Enhance its ability to transform inputs into outputs efficiently
• Recognise the importance of maintaining stable relationships with the environment and meet stakeholder demands
• Note that being effective may involve giving priority to what is best for the cause rather than what is best for the organisation. 

Empirical

But there is often a gap between theory and practice. So we looked for relevant empirical studies on what drives effectiveness. We reviewed 30 academic articles, and this included two literature reviews that covered many more articles.

The first, by Helmig, Ingerfurth, and Pinz (2014) summarised determinants of success and failure – the two extremes of effectiveness – in the non-profit sector found in 152 articles.^[2] (See Appendix 1 for more details about those determinants.)

The second, by Liket and Maas (2015), found 52 organisational characteristics that were predicted to promote effectiveness from their review of over 100 articles (there is some overlap between the articles covered by the two reviews).^[3]

Liket and Maas whittled these 52 characteristics down to 26 after first comparing them with a set identified independently by a group of experts from the non-profit sector in the Netherlands and then testing them with 83 Dutch organisations to make sure the characteristics were clear and relevant to most non-profits. (See Appendix 2 for the final 26 characteristics.)

Significantly Liket and Maas also removed two sets of characteristics that might be thought to be important:

1. Those that could not be easily objectively verified, such as charismatic leadership, willingness to change, ambition, and employee motivation;
2. Three characteristics that were unclear or only relevant to a few organisations: the presence of a logic model, measuring outcomes, and use of external evaluations.

Some characteristics were common to both studies, notably the importance of strategy and governance/ownership, and to some degree cooperation with other organisations. But there were also differences. Helmig et al included financial health and human resource management as important factors. Liket and Maas included characteristics such as: transparency in reporting; responsiveness to stakeholders; use of evidence-based programs or participative design in developing programs; evaluation activities; and consideration of what similar organisations do and alternative ways of achieving the mission.

Many of the good management practices identified by Liket and Maas are similar to those identified by another literature review, Herman and Renz (1998) – such as a written mission statement, measuring client satisfaction, and having a statement of organisational effectiveness criteria, goals, or objectives.^[4]

Most of the characteristics described above relate to how the organisation operates internally. However, aligned with the strategic constituency theory, Balser and McClusky examined how non-profit organisations manage their relationships with stakeholders and how these practices relate to perceived organisation effectiveness (Balser and McClusky, 2005). They note that: ‘while existing literature debates the use of correct procedures and successful outcomes (goal attainment) to indicate organisational effectiveness, new literature…is pointing to stakeholder and external relations management as a source of effectiveness.’

Herman and Renz (2008) made a similar point that organisational responsiveness is a useful organisation-level effectiveness measure.^[5] However, in a subsequent discussion on organisational effectiveness, Herman and Renz expressed the caveat that ‘Non-profit effectiveness is related to the use of correct management practices but not in any simple ‘best practice’ way.’

Our review of the literature was useful in that it identified several important characteristics, namely:

• strategy
• governance / ownership
• cooperation with other organisations
• financial health
• human resource management
• transparency in reporting
• responsiveness to stakeholders
• use of evidence-based programs or participative design in developing programs
• evaluation activities, and
• consideration of what similar organisations do and alternative ways of achieving the mission.

But just looking at the empirical research is limited by:

• The relatively few empirical studies available,^[6] because of limited funding, compared to the very large number of organisations and number of possible characteristics to test.
• Methodological constraints, such as the different conceptions of what effectiveness is, as noted above and the challenges of measuring organisational characteristics such as leadership, skills, and culture, mean that any conclusions from the research will be tentative, not authoritative.
• The lack of consensus of studies and academics on some issues. In particular, a consensus on how to operationalise effectiveness remains elusive.^[7]

These limitations result in part from the limits of applying deductive reasoning in this complex field. Deductive reasoning involves looking at many cases of effective and ineffective organisations, and identifying the characteristics most frequently associated with the effective ones. Knowledge is built up by looking at as many organisations as possible to see which characteristics and combinations keep popping up. This requires a large data set with common measures not only of effectiveness but also of the various characteristics being tested. This is difficult and costly to do.

Experiential

To overcome these limitations, SVA also considered experiential knowledge. In contrast to academics, practitioners rely more on inductive reasoning based on their experience. They develop hypotheses about which characteristics are most important and under what conditions based on a deep review of a relatively few organisations. They refine these hypotheses for every new organisation and identify the emerging patterns. This approach typically gives greater weight to factors that are hard to make objective and measure, such as leadership, culture, ambition, and values.

To capture this type of knowledge SVA consulted two sets of practitioners.

SVA’s experience

The first set was the experience and expertise of current and former SVA staff having helped social sector organisations in Australia for 15 years.

SVA has worked closely with 40 social ventures like Ganbina, STREAT and AIME supporting their growth and delivered $18 million through government and cross-sectorally funded social enterprise programs to support over 60 social enterprises.

SVA’s experience and expertise from this history is ‘baked into’ how the organisation thinks and works…

Since 2007, the consulting team has worked with more than 300 organisations in over 650 projects supporting those organisations in strategy development and implementation, governance, outcomes management and collaboration. SVA has also worked with philanthropists, corporates, trusts and foundations, and government to generate over $86 million of investment into the social sector, and dedicated $10 million to the incubation and development of projects trialling new approaches in education and employment.

Much of SVA’s experience and expertise from this history is ‘baked into’ how the organisation thinks and works, as insights are passed on from staff member to staff member and captured in processes, tools, and methods.

A review of SVA documents and business processes and interviews with current staff members identified several organisational characteristics that SVA believes are key when it comes to promoting effectiveness:

• strategy (vision, mission and goals)
• financial health
• measurement and evaluation / continuous learning
• working for system change
• engagement in partnerships
• strong program design
• governance
• legal and risk management
• understanding of ecosystem
• staff and human resources, and
• good use of and efficient information technology.

These include many of the good management practices identified above from the literature review as well as three – working for system change, engagement in partnerships, and understanding of ecosystem – that are outward-looking.

Effectiveness frameworks from other organisations

The second set of experiences was that of eight other experienced and well-respected organisations that have sought to identify organisational characteristics that promote effectiveness in the non-profit and social enterprise sectors.

Four of them reflect a general consensus of what experienced practitioners consider important:

1. The Leap Ambassador’s Performance Imperative Organisational Self-Assessment (PIOSA)
   A notable feature of the seven pillars of the PIOSA tool is the importance it gives to the authentic intention of the organisation’s leadership to make the organisation as effective as it can be. Also three of the pillars are related to learning, monitoring, and evaluation. SVA was a pilot in the testing of the PIOSA and found it to be very useful. The SVA Fundamentals were heavily influenced by PIOSA.
2. McKinsey’s Organisational Capacity Assessment Tool (OCAT)
   McKinsey’s OCAT is one of the most well-known and used of tools designed to promote effectiveness. It does this by providing an online survey that assesses nine characteristics of an organisation along four levels (1-4), where 1 suggests improvements are needed and 4 suggests the organisation is already strong in this characteristic. The SVA Fundamentals Assessment tool uses a similar four level grading scheme to OCAT.
3. NPC’s Charity Analysis Framework
   Unlike PIOSA and OCAT, NPC’s Charity Analysis Framework is not designed as an assessment tool but provides a set of questions about four elements of an organisation that anyone who is reviewing a charity should consider.
4. Center for Effective Philanthropy’s Definition of Foundation Effectiveness
   The Center for Effective Philanthropy (CEP) has created a definition of effectiveness for funders based on its own experience and research. The challenges identifying determinants of effectiveness for funders, are notably harder than they are for organisations they fund. Two of the four ways of measuring effectiveness – attracting resources and attaining goals – are not easily applicable to funders who typically do not have to attract resources themselves and who rarely assign themselves specific, measurable goals to attain.

The other four were checks on what we had gathered so far, and were used in minor ways in developing the tool:

• Norton and Kaplan’s Balanced Scorecard
• Social Ventures Partners’ Organisational Capacity Assessment Tool
• Marguerite Casey Foundation’s Organisational Capacity Assessment, and
• TCC group’s Foundation Core Capacity Assessment Tool.

As shown in Appendix 3, there is much overlap with the characteristics identified in the literature review, such as strategy, evaluation practices, financial health, and governance. But, as expected, the characteristics identified by practitioners give more weight to nebulous, hard-to-measure factors such as leadership, culture, ambition, and values.

SVA Fundamentals for Impact

In combining the findings from all three sources described above, SVA identified 15 characteristics and divided them into three groups that are intuitive and reflect SVA’s experience and way of thinking:

• Client centred
  • Focus on end beneficiaries
  • Evidence informed appropriate services
• Effectively run
  • Financial health
  • Skills and capabilities
  • Leadership
  • Governance
  • Values
  • Learning culture
  • Measurement and evaluation practices
  • Organisational strategy
  • Mission/vision/ambition
  • Operational efficiency
• Engaged with the ecosystem
  • Understanding of and contribution to ecosystem
  • Networking and collaboration
  • Accessibility and responsiveness to stakeholders.

These 15 characteristics clearly come from the three sources outlined above although two require explanation:

• Focus on end beneficiaries, and
• Understanding of and contribution to ecosystem.

Both are included to counter the risk that organisations become primarily occupied with their internal workings rather than the purpose of their existence, to achieve better outcomes for people in need.

The first, having a ‘focus on end beneficiaries’ can be thought of as an organisational characteristic that cuts across all functions and management practices. As noted above, one definition of effectiveness reflects an organisation’s ability to acquire and maintain resources critical for its survival. This raises the spectre of social sector organisations existing to survive, rather than existing for the social good. This characteristic is included to help keep organisations focused on the people it intends to help.

The second, having an ‘understanding of, and contribution, to ecosystem’ is included in part to address the sub-optimisation problem noted above. The idea is that organisations should be seeking to maximise system-wide, shared goals, not just their organisational goals. The SVA Fundamentals Assessment tool that accompanies the SVA Fundamentals includes several checks for this, but this characteristic makes that expectation explicit.

What makes the SVA Fundamentals different?

Given the research and tools that others have developed and the SVA Fundamentals has drawn from, one may wonder why SVA considered it necessary to develop its own framework rather than adopt an existing one?

While the SVA Fundamentals are not the final word when it comes to identifying organisational characteristics that promote effectiveness, we believe the framework adds to current practice in four ways. The SVA Fundamentals:

1. Are derived from three sources of knowledge: theoretical, empirical, and experiential. Most research and tools give much more weight to one or maybe two of these.
2. Give more emphasis to the role of the organisation within the broader ecosystem than most other frameworks to avoid the problem of creating sub-optimum solutions to pressing problems. The SVA Fundamentals reflect the growing appreciation that being an effective organisation requires working with others to change the system rather than going it alone.
3. Combine international research with experience of the Australian social sector. The SVA Fundamentals are not only applicable to Australia, but they have been developed with this country specifically in mind.
4. Are explicit about making sure what the organisation does is centred around the needs of the end beneficiary, not the organisation. It is all too easy for an organisation to focus on itself rather than keep the needs of those who it intends to help at the forefront.

‘All models are wrong, but some are useful.’

Finally, a note of caution. There is a difficult balance to achieve. On the one hand the research and practitioner experience show that there are certain characteristics that are important to have if you are an organisation that wants to bring about positive social impacts. But it is equally clear that the research and experience show that which are most important, when, and how they are important differs across time, place, and sector. The SVA Fundamentals for Impact is a framework, not a formula, for effectiveness.

As such, the words of George Box, the famous statistician, very much apply to the SVA Fundamentals: ‘All models are wrong, but some are useful.’

About David Pritchard

David is the former Head of Measurement and Evaluation for London-based New Philanthropy Capital and now advises non-profits, social enterprises and foundations in the UK, EU and US on how to assess and increase their social impact. He is currently Adjunct faculty member of non-profit management at Adler University, Co-founder and President of Social Value USA, Director of Impact Measurement and Program Evaluation at CAF America, and Chief Impact Officer at Quantified Ventures, a financial intermediary that supports social impact bonds.

SVA engaged David to take the lead on developing the SVA Fundamentals for Impact due to his extensive track record in the sector for rigorous and thoughtful innovations.

Find out more about the SVA Fundamentals for Impact here.

Notes

[1] Helmig, B, Ingerfurth, S., & Pinz, A. (2014). Success and Failure of Nonprofit Organizations: Theoretical Foundations, Empirical Evidence, and Future Research.  Voluntas 25:1509–1538

[2] Helmig, B, Ingerfurth, S., & Pinz, A. (2014). Success and Failure of Nonprofit Organizations: Theoretical Foundations, Empirical Evidence, and Future Research.  Voluntas 25:1509–1538

[3] Liket, K.C, & Maas, K. (2015). Nonprofit Organizational Effectiveness: Analysis of Best Practices. Nonprofit and Voluntary Sector Quarterly 44(2) 268–296

[4] Herman, R.D. &  Renz, D.O. (1998). Nonprofit Organizational Effectiveness: Contrasts Between Especially Effective and Less Effective Organizations. Nonprofit Management & Leadership, vol. 9, no. 1, Fall.

[5] Herman, R.D. &  Renz, D.O. (2008). Advancing Nonprofit Organizational Effectiveness Research and Theory, Nine Theses. Nonprofit Management & Leadership, 18(4).

[6] Lecy, J.D, Schmitz H.P.,& Swedlund, H. (2012). Non-Governmental and Not-for-Profit Organizational Effectiveness: A Modern Synthesis. Voluntas 23:434–457.

[7] Lecy, J.D, Schmitz H.P.,& Swedlund, H. (2012). Non-Governmental and Not-for-Profit Organizational Effectiveness: A Modern Synthesis. Voluntas 23:434–457.