ADD2642 Evaluation of the London Multiply Programme

1.1.    The London Multiply programme is a circa £41m numeracy skills programme that supports free learning opportunities for Londoners who are aged 19+ and have no to low numeracy qualifications. The programme is being delivered by local authorities and colleges working in partnership with employers, schools, and community organisations, and will support a minimum of 31,000 Londoners to improve their numeracy skills across 57 providers. As a condition of funding from the Department for Education (DfE), Multiply provision is not permitted to duplicate existing Adult Education Budget (AEB) provision.
1.2.    To ensure the programme delivers value for Londoners, it is important for the GLA to understand the impact of the programme on participants, including social and economic outcomes. It is also important for the GLA to understand the effectiveness of the London Multiply programme in reaching those outcomes, especially the significant engagement activity being delivered, so that future provision of adult skills and training can incorporate best practice.
1.3.    The GLA will analyse quantitative data that will be able to ascertain some outcomes of the programme. However, an external evaluation is also needed to gain a more robust and in-depth understanding of the London Multiply programme and its impacts on Londoners, wider stakeholders and delivery partners. 
1.4.    MD3035 (Management and Administration of the 2022-24 Multiply Programme) delegated authority to the Assistant Director – Skills & Employment to make all further programme-level decisions via ADD Form as necessary. It was noted within the MD that the GLA would conduct an evaluation of the London Multiply programme (using up to £210,000 funding) that would focus on how the programme meets local priorities such as learner progression. This decision form approves the final scope and budget for that evaluation. 
1.5.    The GLA will conduct an open procurement exercise to commission an evaluation organisation in line with the GLA Contracts and Funding Code. This process will ensure that the successful evaluation organisation will offer suitable value for money, specialist knowledge, and experience of working with the adult education sector.
 

2.1.    The GLA will procure an external evaluation organisation to conduct an evaluation of the London Multiply programme to understand how well the programme met the intended outcomes, helping to meet the Mayor’s ambitions to make adult learning in London more impactful. As the programme is supporting new activity that does not duplicate existing AEB provision, the evaluation will also consider what effective new practice could be incorporated into the Mayor’s skills policy and funded provision, including AEB, in the future.
2.2.    Through the evaluation the GLA aims to :
•    understand London Multiply numeracy provision, what activity was delivered and where, how it was different to existing adult numeracy provision, the impact of provision, and how it can be embedded into future programmes
•    understand the economic and social outcomes for learners who participated in London Multiply provision
•    evaluate the effectiveness of varied outreach approaches to engaging those with attitudinal barriers to maths and those with low numeracy confidence, including whether the programme has worked to address those attitudes
•    understand the impacts of the programme on community organisations and employers, referral pathways that have been created, the impact of these, and how likely they are to be sustained beyond the life of the programme
•    understand successful practice that can be integrated into ongoing policy development and skills delivery, to ensure successful approaches are built upon and support a legacy of good numeracy learning in London.
2.3.    The London Multiply evaluation will include the following elements for which the GLA will seek an external evaluation organisation at a cost of up to £150,000:
2.3.1.     Qualitative research of provision and outcomes, which will include: 
•    Analysis and individual case studies that reflect the experiences of learners on the London Multiply programme.
•    Analysis and case studies of how providers are delivering the London Multiply programme, how their interventions differ from existing provision, the lessons they are learning from the London Multiply programme, and the best practice that they will continue beyond the programme.
•    Analysis and case studies of how supporting partners, such as community organisation and employers, have been involved in London Multiply, their experiences and impact of the programme, lessons for future cross-sector partnership working, and how likely they are to sustain relationships established through the programme. 
2.3.2.    Evaluation of the Local Community Numeracy Roadshow that will highlight reach, participation levels, impact, effectiveness in reducing attitudinal barriers to numeracy, and learning on local engagement.
2.3.3.    Final programme evaluation report synthesising the findings from each strand of evaluation work, including:
•    Relevant learnings and data from the evaluations the Skills for Londoners (SfL) Community Outreach programme and SfL Employer Engagement programme both supported by London Multiply funding. 
•    Relevant learnings and data from quantitative analysis conducted by the GLA.
•    Policy and practice recommendations for the GLA and providers on how to sustain best practice from London Multiply beyond the life of the programme. 
2.4.    Quantitative analysis and commentary of the Individualised Learner Record (ILR), the London Learner Survey (LLS), and regular reporting from London Multiply providers will be conducted by the GLA. This analysis will be incorporated into the final programme evaluation report delivered by the commissioned evaluation organisation.
 

3.1.    To ensure the London Multiply programme meets the Public Sector Equality Duty an Equalities Impact Assessment (EqIA) was carried out to demonstrate how equality groups and equalities impact were considered in the development of the programme and set out the likely impact delivery would have on equality groups and address wider issues of socio-economic inequality.  
3.2.    The London Multiply programme will give Londoners without Level Two numeracy qualifications the opportunity to learn and develop essential numeracy confidence and skills they need in life and work. From the EqIA, the GLA knows those without a Level Two qualification are more likely to be low-income, older, disabled, or from a minority ethnic background. As such, this provision is likely to reach groups sharing a protected characteristic under the Equality Act and support them to upskill, supporting progression into further learning and good work. 
3.3.    As a result of this focused work to support equality groups, it is expected that any evaluation of London Multiply will pay particular attention to understanding the impacts of the programme on different equality groups and their ability to access adult numeracy training.
3.4.    The evaluation is also seeking to explicitly incorporate the voices and experiences of Londoners from equality groups through the qualitative research and evaluation of the Local Community Numeracy Roadshow. This will ensure that those most likely to face inequalities in education and skills are represented strongly in the evaluation of adult numeracy provision in London.
3.5.    The outcomes of this evaluation will also be used to ascertain best practice and adapt future delivery of adult education in London to ensure that it is more accessible to those less likely to participate in adult education, including those from equality groups.
 

4.1.    A competitive exercise will be conducted, in line with GLA and TfL procedures, to ensure that the procured evaluation organisation will offer suitable specialist knowledge and experience of working within the adult education sector, which will offer unique value for money.
4.2.    The budget for this activity is £150,000. This has been benchmarked against other evaluations conducted on behalf of the GLA, including the evaluation of the AEB.
Key risks and issues:

Links to Mayoral strategies & priorities
4.3.    The Skills Roadmap for London outlines the Mayor’s strategy for ensuring that the learning that City Hall funds makes an impact and leads to positive economic and social outcomes.
We want the learning we fund to transform people’s lives, making an impact and leading to positive economic and social outcomes. 
4.4.    This evaluation will support this priority by measuring the social and economic benefits of the London Multiply programme, which will give the GLA greater understanding of best practice and interventions that support Londoners to develop the essential numeracy skills and confidence they need for everyday life, to progress in learning, and access good work.
4.5.    This evaluation will also support the Mayor in making skills provision more locally relevant.
Making skills provision more locally relevant means providing a more joined-up skills and employment offer that meets the needs of Londoners and the local economy, including businesses and employers.
4.6.    This evaluation will directly support this priority by giving insight into how adult education providers in London have used the London Multiply programme to build partnerships and collaborative work with local organisations, and how they have supported learners to progress into further learning and good work. This insight into best practice will be integrated into London’s skills provision, including Adult Education Budget (AEB) numeracy provision, going forward.
4.7.    There are no conflicts of interest to declare from those involved in the drafting or clearance of this decision.
 

5.1.    This Assistant Director Decision (ADD) seeks approval for the expenditure from the London Multiply programme budget of £150,000 for externally commissioned elements of the London Multiply evaluation (£50,000 in FY 2023-24 and £100,000 in FY 2024-25).
5.2.    MD3035 noted that up to £210,000 funding would be used for the evaluation of the London Multiply programme and delegated authority to the Assistant Director – Skills & Employment to make all further programme-level decisions via an ADD, as necessary.
5.3.    The split of the expenditure by FY is set out in the table below.

5.4.    There is sufficient budget in the 2023-24 financial year to meet the expenditure of £50,000 from the London Multiply Programme budget. 
5.5.    Budget allocation for future FYs (2024-25) will be subject to the annual budget setting process and is subject to change.
5.6.    Any contracts that commit the GLA in future years are subject to appropriate break clauses.
 

6.1.    The next steps are summarised below:

MD3035 Multiply Programme 2022-24