MOPAC evaluations

'Evaluation is an objective process of understanding how a policy or other intervention was implemented, what effects it had, for whom, how and why'

The term ‘evaluation’ covers a wide variety of research techniques each looking to explore specific aspects of a policy or programme. Usually, we look at questions such as whether something has worked, whether something was value for money, or questions around why something happened.

There are many types of evaluation and each type has its own strengths and weaknesses. There is no single ‘right’ way to evaluate, although there are ‘wrong' ways. Sometimes, an evaluation might not be needed at all. For MOPAC, a good evaluation is about more than the types of techniques used – innovation, pragmatism, statistics, usefulness and integrity in reporting are all important considerations for us.

How we choose the type of evaluation depends on the programme or policy we want to study, the main questions we want to try and answer, and other factors such as resource availability and budgets. Some evaluations can be very quick to complete, others can take years and be very resource and time intensive. Good evaluations should always be proportionate in design and cost to the project they are looking at.

Good evaluation is an essential learning tool for organisations – for example evaluations can:

• tell you if something has really worked or not;
• tell you why something has worked or not;
• help understand if something has provided value for money;
• provide good evidence for future decision-making;
• identify learning opportunities or training gaps;
• identify unintended consequences or outcomes of a given policy;
• better communicate the value of a piece of work; and
• help ensure accountability to the public for the work done on their behalf.
  

MOPAC's Evidence & Insight Unit, a team of researchers and analysts with specialist expertise on policing and crime, conducts a wide range of research, including evaluations.

There are four types of evaluation that MOPAC undertakes. Sometimes, our researchers will use more than one type for a project to help ensure we get the best evidence.

Performance monitoring and analytics - what has happened during a project?

This kind of evaluation considers data around the deliverables of a project, including how many people it has involved and how long they were engaged for. This is the most common evaluation approach MOPAC uses. MOPAC researchers also use data to develop detailed profiles of specific crime issues, and also publish a wide range of data for Londoners and colleagues in partner agencies to look at and use via our interactive dashboards.

Process evaluations - how has a project been delivered? What do people think of it?

Typically, this kind of evaluation involves interviews, surveys and focus groups with people involved, observation of the project in action by researchers and diary studies which look at the activities, behaviours and experiences of a group of people over a period of time.

Impact evaluations - has a project worked?

Researchers use a range of robust methods such as randomised control trials and experimental research techniques to see what difference a project has made or not. These evaluations tend to be the most resource and time intensive to carry out and are typically conducted on only the most significant and high value projects.

Economic analysis - was it value for money?

Researchers review a project's budgets and spending, and conduct a cost/benefit analysis to understand if it provided value for money.

You can see more about MOPAC's evaluations menu and approaches here.

MOPAC believes in continuous improvement and transparency in the way we evaluate projects.

Commitment to transparency

Our intention is to routinely publish all final and interim evaluation findings. We recognise that there are some risks in publishing interim reports (i.e. the results may be subject to change), and as such we have developed a position statement on the usage and reporting of all evaluation research.

Documenting how MOPAC uses evaluation research findings

Our evaluation process is not only about conducting the work itself, but making sure that the findings are used to help start new conversations and assist in future decision making. Moving forward, alongside published evaluation reports, we are working to provide information outlining how our research has been used.

Sharing MOPAC evaluation expertise

MOPAC have developed a range of tools to help others to evaluate their work and use evidence to make decisions. For example, we provide specific tools and training on evaluation for organisations receiving funds from MOPAC and routinely provide expert advice to other organisations.

We are also pleased to offer our Evaluation Support Tool to any organisations and individuals looking to evaluate a given project. This unique and innovative tool covers the entire evaluation journey, from understanding a problem, to evaluation methodology, through to writing up findings, and provides active support throughout the journey and advice over which types of evaluation are feasible under different conditions.