For many years Social Ventures Australia has worked to identify and support good practice in delivery of employment programs for young people. What we know – and what the evidence shows – is that there is no silver bullet. There are elements that go into good program design like ensuring that young people have the chance to meet employers. But the overwhelming message is that programs must be tailored, both to changing local labour markets and the needs and aspirations of young participants. Youth employment programs cannot be ‘set and forget’. They have to be monitored, evaluated and adapted to make sure that they are actually delivering for young people.
Over the last three years, SVA has been undertaking a project to build the capacity of employment program providers to improve employment outcomes for young people. Our focus has been on programs which are philanthropically funded or developed outside the government system. Many of these programs have been developed to address a perceived gap. But what we have found that many of these organisations lack the time and resources to evaluate or demonstrate exactly how well they are going. They often collect the information required by funders, but gain limited insight into where their programs might be improved.
Through the Review for Outcomes project, SVA has created free tools and resources to help organisations delivering employment programs for young people do just this. The centrepiece of this work is the free online Review Platform, which automates the process of surveying young people about their experience and outcomes, and provides a range of dashboards to track key indicators. SVA is working with Review users to help them identify insights from this data and to use them to drive improvements. In the process we are learning more about the sorts of challenges young people accessing these programs face.
Snapshot of SVA’s journey with Review
Our starting point for this project was a review of the latest research into what works in employment programs for young people. What the international evidence suggests is that most programs that work with young people have little or no impact (Kluve et al 2016). However, those that are more successful use a multifaceted approach that is tailored to the needs of participants and delivered close to the ground (Kluve et al 2016, Borland et al 2016). The details of program design, and their implementation, are critically important to success. So, guided by an expert reference group which included academics and practitioners, we set about identifying the common program elements that work together to deliver better outcomes – in other words, the building blocks of a ‘multifaceted approach’. We distilled these into a list of ten:
Not all programs have the scope to be able to deliver each of these elements directly, and might need to tap into a wider ecosystem of services and support. But it is important to recognise that the system works best when all the elements are in place. For example, a lack of attention to what supports are available to participants following the program increases the likelihood that young people will not achieve their employment goals.
The poor success rates of youth employment programs overall means that it is even more important than usual to understand what impact they are having. If the best employment programs are highly tailored to their participants, then the best source of information about whether they hit the mark is the participants themselves. When we consulted with practitioners about what they were doing to understand their performance we found that very few had systematic approaches to collecting information from the young people they worked with. Most collected data for reports to funders, but lacked the resourcing to develop, administer and analyse feedback from participants.
To address this gap, SVA worked with a group of organisations over 2019/20 to develop and test an online platform that makes it easier for organisations to gather and use feedback from young people on their experiences both in and following their programs.
In June 2020, the Review Platform was launched.
The Review Platform – Overview
The Review Platform automates the process of administering and collating survey data from program participants.
Embedded in the Platform are three surveys which are completed by participants at program start, at its end, and some time (usually three months) after completion. Organisations can upload participant information and set survey dates. The system sends out surveys via email or text and represents survey data in a series of dashboards. All survey information is confidential.
The surveys at the heart of Review have been designed to gather three types of information:
- Basic demographic information, including obstacles to finding employment;
- Changes in employment/education status;
- Participant experience of the program relevant to those elements of youth employment programs that are likely to contribute to success.
For example, participants are asked whether there was someone they trusted for help or advice during the program, and whether they had the chance to do work experience. Responses to these and other questions are used to provide a snapshot of strengths and weaknesses in each of the ten areas outlined above that were identified as contributing to success.
Platform users can also see changes over time in key areas that are important to employment prospects.
Importantly, given the challenges young people face finding work that meets their needs, the survey collects information about job quality including both objective indicators (such as ‘casual’ or ‘ongoing’ employment) and subjective indicators like ‘job satisfaction’.
Experience so far
Twelve organisations, with youth programs ranging in size from 15 to over 200, have started to use the platform since launch. While many have experienced significant delays and disruptions related to Covid-19, they (and we) are already seeing value in the information being gathered.
For example, one organisation noticed that its caseload of young people from Culturally and Linguistically diverse youth had grown by 20% in 12 months. It used Review survey data to understand the specific needs of this group. Their analysis led to the development of a specialist program with an increased focus on navigating job search portals, additional industry visits and guest speakers to improve their knowledge of the local labour market and expand their industry network.
While designing a new program for sole parents, another provider realised that they had made some incorrect assumptions about the employment challenges facing their cohort, including access to networks and levels of literacy and numeracy. These insights, and further discussion with participants, led to a re-shaping of program content, including language, literacy and numeracy training.
For some, the Review Platform has increased their ability to prove their impact to stakeholders and potential funders:
“Review is a powerful tool that has the potential to support and actively improve a program’s long-term sustainability and impact. It has saved us significant time and money.”
Program Manager of a social enterprise using Review
The same enterprise reported that use of the surveys had strengthened their relationship with participants, who saw that their views were valued by the organisation.
What else have we learned?
While the data gathered through the Review platform cannot be generalised to the broader youth population, it is providing insights into the common challenges faced by organisations trying to help young people find work, and helping us consider what else we can do to support them.
We can see, for example, that over half of the young people accessing these programs left school before Year 12. Those who identify themselves as having literacy or numeracy challenges are the least likely to find work following their employment program. We can also see that many young people have identified family relationships as one of the major obstacles to them finding employment. As expected, those who are able to find full time work or combine work and study are those most likely to be satisfied with their work.
Where there are common challenges, there is an opportunity for us to bring organisations together with other experts, including young people, to talk about what else can be done.
The work of ensuring that young people can move into good quality jobs is more important than ever. While we know what elements to look for in program design, understanding the experience of young people is essential to maximise impact. Non-profit organisations, particularly those operating on a small scale, can capture substantial benefits by using the Review platform to gather ongoing participant feedback. As the number of participating organisations grows, we can get a clearer picture of the opportunities to highlight shared challenges and support the sector to be the best that it can be.
Talk to us about Review. SVA is continuing to support organisations to join the platform. If you or someone in your network is interested please contact the Project Lead Trent Miller on 0430 675 393, or email firstname.lastname@example.org. We also welcome discussion and feedback about the project itself.
We acknowledge the commitment and support of Citi Foundation who have been the major funder of the Review project over three years. We would also like to thank the Lord Mayor’s Charitable Foundation, The Jack Brockhoff Foundation, Chris & Gillian Lee and The Osborne Family for their generous support for this work.