ADD2092 Impact evaluation in the Education and Youth team

Type of decision: 
Assistant Director's decision
Code: 
ADD2092
Date signed: 
20 March 2017
Decision by: 
Amanda Coyle, Assistant Director, Health and Communities

Executive summary

Project Oracle will provide training and support to the Education and Youth team to ensure measuring impact is embedded in all of our programmes. This will be particularly beneficial over the next few months as new programme areas are established. Project Oracle will provide a tailored support package.  

Project Oracle will also work to build evaluation into the Careers Clusters programme. 
This activity will be funded through the Project Oracle budget.

 

Decision

That the Assistant Director of Health and Communities approves:

1.    £15,000 to cover the evaluation training and support (delivered by Project Oracle) required for the Education and Youth team.
2.    £3,000 to cover the training and support (delivered by Project Oracle) required for the careers clusters programme. 
 

Part 1: Non-confidential facts and advice

Introduction and background

Project Oracle’s expertise lies in getting youth organisations and funders to produce, use and share high-quality evidence, so that they can make better decisions to improve how interventions are funded and delivered.

Project Oracle has been funded by the GLA in partnership with the Mayor’s Office for Policing and Crime and ESRC for over 3 years.

The GLA has a current Grant Agreement in place with Project Oracle for £125,000 (from 1st August 2016 to 31st July 2017). This was approved through MD 1607 and MD 1260 approved the funding from 2013 to 2016 with funding partners MOPAC and the ESRC.

Project Oracle’s 2013-16 grant agreement saw an under delivery of £40,000 (of which £17,142.86 is the GLA’s contribution).  This funding will be utilised to undertake this additional activity for delivery by Project Oracle. 
 

Objectives and expected outcomes

The Education and Youth team works across the whole of London and our programmes have reached over 1,700 schools.  The Education and Youth team therefore have the ability to make a big impact in the sector and it is vital the correct procedures are in place to measure impact.  

Our aims
-    To have a clear understanding of what we want our programmes to achieve 
-    For the Education and Youth team and the career clusters to feel more confident about evaluation 
-    To have systems in place to appropriately measure the impact of our programmes 
-    To establish evaluation of impact as a core element of our work. 

As a result we expect to see the following outcomes:
-    Increased engagement of schools/teachers/young people with our programmes 
-    Our programmes more generally achieving our aims 
-    Better value for money 

Project Oracle will undertake bespoke workshop session across the Education and Youth programme areas to establish theories of change and associated evaluation activity to demonstrate impact.  A similar programme will be undertaken for the Careers Clusters. They will also be undertaking Project Oracle validation (this will be funded separately by each Careers Cluster). 
 

Equality comments

The 2016 Mayor’s Annual Education Report sets out an overview of London’s education system. It covers the progress of children and young people from early years to key stage 5 (post 16). This identifies a number of key challenges across the education system including equality issues. These are also reflected in the Mayor’s equalities framework.  

The proportion of London’s 5 year-olds achieving a good level of development has been increasing rapidly in recent years and now nearly three-quarters (71.2 per cent) of children achieve this threshold (cf national average of 69.3 per cent). Where London has performed less well at early years is on take-up of both the two and the three to four year old offer. There is year-on-year improvement, although still below the national average, but with wide variation across London. 

London continues to do well across primary and secondary school performance measures particularly for the most disadvantaged. In 2016, 57 per cent of pupils in London met or exceeded the new expected standard in all three of reading, writing and maths by the end of Key Stage 2, compared to a national average of 52 per cent. At Key Stage 4, London had the highest Attainment 8 score of any region in 2016 (51.7 points compared with a national average of 49.9).  

The gap between disadvantaged children and their peers is lower in London than elsewhere in the early years, at the end of primary school and at the end of secondary school. However, the attainment gap increases with a child’s age and at the end of secondary school is around a third larger than at primary school.  The disadvantage gap in London is evident across all ethnic groups but is widest for white pupils at both Key Stage 2 and Key Stage 4. At both Key Stages, white disadvantaged pupils are the lowest performing group followed by pupils from black and mixed disadvantaged backgrounds. 

The overwhelming majority of London’s 16 year-olds continue in education after GCSEs. Relatively few young people in London go on to apprenticeships straight after GCSEs: 3 per cent in outer London, 2 per cent in Inner London compared to 6 per cent in England. By age 19, nearly nine in ten (88 per cent) of London’s young people are educated to level 2 – the equivalent of five GCSEs at grades A*-C, slightly above the England average, and nearly two thirds hold the equivalent of two A-levels. The variation across the city is striking, in Harrow three-quarters of young people are educated to level 3, in Barking and Dagenham nearly half are not.

The purpose of the focus on evaluation across the education programme is to ensure that the Mayor’s planned education programmes will undertake activity to address these issues.  

The Mayor has also made a number of pledges which are specific around equalities outcomes (see below). 

This support provided by Project Oracle will work with all members of the Education and Youth team and Careers Clusters providers so that individual programmes should have equality considerations built in them to ensure they are not further disadvantaging vulnerable groups. We will also consider this in our evaluation practices. 
 

Other considerations

Risk

Impact

Likelihood

Mitigation

We have not allocated sufficient money for this piece of work.

High

Low

We have received a proposal from Project Oracle that is under budget.

Once we have our initial needs assessment completed Project Oracle are prepared to make amendments to the proposal accordingly.

 

The Education and Youth team do not engage with this project.

High

Low

I have had conversations with the team and there is a clear interest in being able to evaluate programmes better.  

Project Oracle are not able to deliver the task.

High

Low

Project Oracle are specialists in this area.

They have experience of delivering similar pieces of work.

 

 

 

 

 

a)    Links to strategies and Mayoral and corporate priorities

This project aligns with the Mayor’s pledges: 

“Put an open data strategy at the heart of London government, with a new London data office working to bring data from across London’s boroughs and public agencies together, and opening it up to enable quicker decision making, better services, more efficient government, and greater transparency”

“Make gender equality a focus of Skills for Londoners, creating targeted opportunities for girls to excel in STEM skills in particular”

“Work with schools to improve careers advice and to promote opportunities for children to get an early start in career skills such as coding or engineering – and making sure all such advice and programmes are gender blind, creating equal opportunities for boys and girls.”
 

 

 

Financial comments

Approval is being sought to approve expenditure of up to £18,000 for Project Oracle to provide a bespoke training and support package to both the Education and Youth team (£15,000) and the Careers Cluster (£3,000). It is anticipated that the full expenditure of £18,000 will take place in the current financial year 2016-17 and will be funded by the Education and Youth Team budget. 

 

Planned delivery approach and next steps

Education and Youth Programme estimated costs and work programme

Activity

Detail

Days

Cost

Milestone

Needs assessment

8 x 1 hour meetings with Project Managers

5

£3,000

End Apr 17

Preparation, interview & write up

Introductory meetings

8 x 2-3 hour meetings with team members as deemed appropriate by Project Manager

4

£2,500

End May 17

Preparation and meetings

Theory of change

0.5 day workshop:

1.5

£1,000

End Jun 17

Preparation & facilitation

Evaluation planning

0.5 day workshop:

1.5

£1,000

End Jun 17

Preparation & facilitation

Follow up meetings

Total time available

8

£5,000

End Jul 17

Preparation of case study

Writing summary report

2

£1,500

End Jul 17

Lesson learning event

Meeting of team members and other relevant stakeholders

2

£1,500

Mid Sep 17

Preparation, meeting and report

TOTAL

 

24

£15,000

 

 

Career Clusters

Activity

Detail

Days

Cost

Milestone

Theory of Change training

Training for Skills Funding Agency contract managers prior to Learning Event – including  Theory of Change as contract management tool

1

£7500

End April

Lesson learning event

Sharing best practice between cluster members:

 

2

£1,500

Mid May

Summary report from validations

Lesson learning for GLA from the validation component of the Careers Clusters

1

£750

End May

Total

 

4

£3,000

 

 

Activity

Timeline

Procurement of contract

February/ Early March 2017

Delivery Start Date

10th March 2017

Career Clusters - Summary Report from validations

End May 2017

E&Y Lesson Learning Event

Mid Sept 2017