An Evaluation of Hospital Youth Workers

You requested:

Dear Colleague, please can you help my understanding of the above report which was done by Evidence and Insight (E&I) Unit - MOPAC’s in-house social research and analytical team.

This is very insightful. I have some questions which I can not seem to garner from the report.

1. Are the authors on the report are employee's of the MOPAC?

2. The report has raised a need for robust evaluation and thus "there continues to be no robust evidence base for the wider approach", (p2) is that suggestion that Hospital Based Youth Workers is not recommended?

3. Regarding staff contacted at the hospitals, those who were interviewed? Numbers appear small, why was this? Considering the projects were in operation for two years.

4. How did Covid affect the project? During lockdowns did youth workers have access to the ED?

5. There appears to be very minimal feedback gathered from client users, young people and parents/carers. Has that affected the evaluation?

6. Throughout the report, it is mentioned that young people withdrew service before completion. What does this mean? For example, p2, "the proportion of young people 'completing' the service was similar". Was there a timeframe of engagement set? That is was it the two years?
If so how was this decided and by whom?

7. Could you please send me documentation of how levels of risk where assessed?

Our response:

1. Are the authors on the report are employee's of the MOPAC?

All authors were MOPAC employees at the time of the report publication, some are still are with the organisation.

2. The report has raised a need for robust evaluation and thus "there continues to be no robust evidence base for the wider approach", (p2) is that suggestion that Hospital Based Youth Workers is not recommended?

This is not suggesting that hospital-based youth workers are not recommended, it is stating there is no robust evidence the approach has an impact on its stated outcomes. The report details the model has promising aspects, but suggests future delivery focus’ on data capture, access and quality to achieve a robust impact assessment to understand which aspects of the service are working and for who.

3. Regarding staff contacted at the hospitals, those who were interviewed? Numbers appear small, why was this? Considering the projects was in operation for two years.

Hospital staff were asked for feedback in two ways – 29 decided to answer a voluntary survey, so this was supplemented by qualitative interviews and focus groups, of which just over 30 were conducted. Sampling was dependant on service providers sending out details for voluntary participation. To ensure GDPR compliance hospital staff contact details were not shared. Given covid restrictions this sample size was considered fair for adding context to service provider performance data and learning themes were consistent, however obtaining more would always be desirable.

4. How did Covid affect the project? During lockdowns did youth workers have access to the ED?

As detailed on page 21 Covid did impact delivery and limited the access youth workers had to the hospital space.

5. There appears to be very minimal feedback gathered from client users, young people and parents/carers. Has that affected the evaluation?

MOPAC research and evaluations always endeavour to include user voice in their products, it is seen as vital rather than nice to have. However, obtaining participants is sometimes a challenge, particularly for this service when the young person has a limited time to engage and maybe reluctant to even receive service support. Whilst achieving additional feedback would be desirable, incorporating any learning from the young people is seen as valuable.

6. Throughout the report, it is mentioned that young people withdrew service before completion. What does this mean? For example, p2, "the proportion of young people 'completing' the service was similar". Was there a timeframe of engagement set?
   If so how was this decided and by whom?

On page 13 & 15 there are service throughput flow diagrams to show the numbers of those who withdrew before completing the service (399 people). This is a category defined and used by service providers themselves, when a case closes due to the young person withdrawing participation or if it was not a ‘planned closure’. It is therefore decided by the youth workers / case managers. Page 19 details the reasons providers suggested disengagement occurs with most recorded as ‘due to personal choice’, with young people stating they simply decided not to continue.

7. Could you please send me documentation of how levels of risk where assessed?

Risk was assessed by the providers in two ways – through a set of questions relating to measures of overall risk, risk of harm from others, risk of harm to others and risk of harm to self and through a set of self-reported questions for the young person that gave scores for all safety and support network measures, wellbeing measures and both violence exposure measures.