MD3314 Adult Skills Fund Evaluation Procurement 2024-25, 2025-26 and 2027-28

1.1.    The independent evaluation of the first four years of delegation of the Adult Education Budget (AEB) commenced in 2019-20 and will end this academic year with the completion of the evaluation for 2023-24. Following a competitive process in 2020, the research agency, IFF Research was awarded the contract to carry out the independent evaluation of the first four years of AEB delegation (2019-20 to 2023-24) to the Mayor of London. IFF Research are currently completing the final year of evaluation covered by this contract.
1.2.    It is crucial that the specialist services required to enable evaluation of the Adult Skills Fund (ASF), as it is now known, are re-procured promptly so that the successful organisation has the necessary time to recruit learners, providers, and employers for interview, from the last academic year of study. It is also key that any new competitively-selected organisation has time to mobilise to meet the established evaluation milestones. This is particularly important as the new evaluation contract will see the publication date moved to March/April of each year from June/July which will be key in providing more timely insights and will allow potential policy changes that may arise from them to be introduced sooner.
1.3.    The ASF Evaluation includes interviews with external stakeholders: the Association of Colleges (AoC), HOLEX representing adult and community learning providers, and the Association of Employment and Learning Providers (AELP) representing independent training providers. Their enthusiasm and commitment to supporting the GLA’s evaluation has been key and forms an important part of this research, and our understanding of how the ASF is perceived by the Further Education (FE) sector in London.
1.4.    Changes to the evaluation from 2024-25 onwards focus on improving the evidence base for internal policy making and ongoing management of the ASF. We will procure a supplier with a high level of FE policy expertise and experience of evaluating theories of change and, propose the inclusion of additional research topics to be formulated each year based on topical issues.
1.5.    The addition of a different topical research questions each year will make the approach to evaluation more agile in responding to issues that occur in-year and cannot be predicted at the start of the contract. It will also remove the need to procure separate research, saving time and reducing the overall cost of research.
 

2.1.    A number of improvements will be implemented for the new procurement:
•    Additional topical deep-dives decided on a yearly basis depending on issues that arise or changes that occur in-year. This could include early performance of a new programme or policy where the GLA may want to monitor perceptions from stakeholders and react to issues quickly such as new funding and accountability reform rates; future scoping of opportunities for the GLA; or the effect of a national policy. The addition of this new topical deep dive will allow the GLA to be agile in its approach to evaluation, garnering key insights to support swift decision-making (if needed) without the need to procure a separate research partner to conduct the work.
•    Appointing a supplier with a high level of FE sector and skills expertise to better put insights into the wider context and policy landscape in and outside of London. This will improve the independence of the report, and increase accountability, whilst also producing more actionable recommendations for improvement.
•    Earlier publication meaning insights from the evaluation can be used to inform evidence-based policy decisions a year earlier than before. The report has ordinarily been published in June/July, a year after the academic year ends. We are proposing to move the publication to March/April, this will mean insights can be used to inform the next academic year, rather than waiting for another academic year, when insights may not be as relevant.
2.2.    GLA officers estimate that the expenditure for conducting the ASF evaluation will be up to £90,000 per year. This estimate is based on the amount needed to attract high quality bids from organisations with FE, skills and employment expertise who will be able to put findings into the wider sector and labour market context. This budget also reflects the inclusion of the new topical deep dives each year and the general increase in the cost of services and inflation since the research was initially procured for the 2019-20 academic year.
2.3.    To provide efficiency of scale and reduce time spent unnecessarily on running procurement competitions, officers propose to procure evaluation services for up to three-years covering the 2024-25, 2025-26 and 2026-27 academic years where the initial period of the contract will be two years, with the possibility of extending for a third year with Mayoral approval via the annual ASF Financial Management decision.
2.4.    Annual updates on the findings of this evaluation will be provided to the ASF Mayoral Board for information.
 

3.1.    Section 149(1) of the Equality Act 2010 provides that, in the exercise of their functions, public authorities, of whom the Mayor is one, must have due regard to the need to: 
•    eliminate discrimination, harassment, victimisation, and any other conduct that is prohibited by or under the Equality Act 2019
•    advance equality of opportunity between persons who share a relevant protected characteristic and persons who do not share it
•    foster good relations between persons who share a relevant protected characteristic and persons who do not share it.
3.2.    Relevant protected characteristics are age, disability, gender re-assignment, pregnancy and maternity, race, religion or belief, sex, and sexual orientation. 
3.3.    In the Skills for Londoners Strategy and subsequent analyses, the GLA has highlighted that there are several groups that are disproportionately underrepresented in London’s labour market including some groups with protected characteristics. This includes Londoners with special educational needs and disabilities (SEND), young people, disabled adults, Black, Asian and minority ethnic groups and women. 
3.4.    The GLA uses the London Learner Survey (LLS) to recruit learners for qualitative interviews as part of the ASF evaluation. Demographic variables are used to ensure a wide range of Londoners are included in the research. The LLS is weighted to be representative of all ASF learners and its methodology includes monitoring the number of responses by protected characteristics of sex, age, ethnicity and LLDD (SEND) status, actively targeting groups that are under-responding with telephone interviews. The LLS is also available in a number of different languages which are reviewed on an annual basis, paper copies are available and there is an alternative version online for those with learning difficulties.
3.5.    This evaluation also uses GLA published analysis of Individualised Learner Record (ILR) data. This published GLA report includes analysis by these demographic variables to increase our understanding of all ASF learners. 
3.6.    The Public Sector Equality Duty has been taken into account by ensuring that all those with protected characteristics are able to take part in the research. Actions to this end include appropriate adjustments offered to interview participants and incentives for their time. 
 

Key risks and issues
4.1.    The risk of delaying the procurement of a supplier for the ASF evaluation is substantial given the established timelines for the evaluation and the need to recruit interview participants whilst their experience of adult education is still fresh.
4.2.    The ASF evaluation so far covers every year of delegation of the ASF, a break in evaluation would create gaps in our understanding of the performance of the ASF. This would particularly affect the qualitative aspect of the evaluation as these insights cannot be captured at a later date. Moreover, the first year of the proposed new evaluation 2024-25 has already commenced.
4.3.    One of the key features of this evaluation is its ability to return insights in a timely manner so that GLA officers can consider potential policy decisions or adjustments. This relies on the evaluation starting on time and meeting its milestones.
Links to Mayoral priorities
4.4.    The ASF evaluation has been developed to provide independent insights on the performance of policies implemented since delegation. This will be key in supporting our understanding of progress against strategic aims as set out in programme mandates and the London Growth Plan. 
4.5.    The Skills and Employment Unit is responsible for the Mayor’s proposed Support Londoners to Benefit from Growth programme. Evaluation of the management of the ASF will be crucial in enabling the unit to understand performance against objectives set through this programme. 
4.6.    The insights garnered from this evaluation will be used to improve performance in future years and enable the GLA to evidence the success of ASF in London to support the case for further devolution. 
4.7.    The evaluation as outlined above has been adapted to provide more flexibility, creating space for ad hoc and emerging issues that GLA officers may need to understand should the national policy landscape change. 
Data protection
4.8.    There is an established process for implementing data sharing agreements between the GLA and third-party organisations carrying out research tasks on behalf of the Skills and Employment Unit. Along with the data sharing agreement, this process requires an information security questionnaire (ISQ) and other relevant information about the third party’s data protection protocols, and all researchers to have DBS checks conducted within the last two years. This is reviewed and updated on an annual basis. 
4.9.    Those interviewed as part of the qualitative part of this research will have provided consent to be contacted for research purposes. 
4.10.    The GLA complies with all requirements of GDPR UK whilst carrying out this work. 
Conflicts of interest
4.11.    There are no conflicts of interest to declare from those involved in the drafting or clearance of this decision.
4.12.    The decisions set out in this form were considered by the ASF Mayoral Board on 10 December 2024.
 

5.1.    The Mayor is asked to approve expenditure of up to £90,000 from the Adult Skills Funding (ASF) Management and Administration budget per academic year to deliver the ASF Evaluation in 2024 25, 2025-26 and 2026-27 academic years, which would bring the total expenditure to £270,000 across three years.
5.2.    This expenditure will be funded from the Adult Skills Fund programme budget across 2025-26, 2026 27 and 2027-28 financial years.
5.3.    Funding for future financial years will be subject to the GLA’s annual budget setting process and annual ASF funding confirmation from the DfE and is subject to change. 
5.4.    Any contract that commits the GLA in future years will be subject to appropriate break clauses. 
 

6.1.    Section 39A of the Greater London Authority Act 1999 permits the delegation of ministerial functions to the Mayor, subject to certain limitations and conditions. This forms the basis of the delegation to the Mayor of AEB functions from the Secretary of State for Education. A particular limitation of the delegation is that the usual power of delegation by the Mayor is not available in respect of section 39A delegated functions. This decision concerns the exercise of those delegated powers.
6.2.    In taking the decisions requested, the Mayor must have due regard to the Public Sector Equality Duty; namely the need to eliminate discrimination, harassment, victimisation and any other conduct prohibited by the Equality Act 2010, and to advance equality of opportunity and foster good relations between persons who share a relevant protected characteristic (age; disability; gender reassignment; marriage and civil partnership; pregnancy and maternity; race; religion or belief; sex; sexual orientation) and persons who do not share it (section 149 of the Equality Act 2010). To this end, the Mayor should have particular regard to section 3 (above) of this report. 
6.3.    If the Mayor makes the decisions sought, officers must ensure that:
•    the services required are procured by TfL Procurement in accordance with the GLA’s Contracts and Funding Code and contracts are put in place between and executed by the GLA and contractors before commencement of the services
•    no reliance is placed on, nor commitments made in reliance of unconfirmed third-party funding and/or future budgets which remain subject to the outcome of the budget setting process for future financial years.
 

7.1.    Next steps set out below: