KEEP Standard Evaluation Report

This evaluation carried out by Ipsos was the first independent evaluation of KEEP Standard in England, informing its effectiveness in a UK context.

4 November 2016

Public opinion

Background

KEEP Standard is a 16-week group training programme for foster and kinship carers, approved to care for children aged 5-12 years old, which aims to improve the skills and confidence of carers when responding to children's difficult behaviour and thus improve child behaviour and carer well-being. Developed in the USA in 1996, KEEP Standard became available in England in 2009 through the National Implementation Service (NIS). This evaluation carried out by Ipsos was the first independent evaluation of KEEP Standard in England, informing its effectiveness in a UK context.

Aims and objectives

The aim of the evaluation was to examine whether carers who completed the KEEP Standard training benefitted from improved outcomes compared with a control group of carers who did not complete the training, using a propensity score matching design on validated scales (SDQ, Parenting Scale, WEMWBS).

Evaluation

This evaluation included data from 12 KEEP Standard groups from ten local authorities in England, which started between September and November 2015. In total, 59 carers who participated in KEEP Standard (the intervention group) completed the measures before and after the training and 26 carers who did not participate in KEEP Standard (the control group) completed these measures with a similar time lapse between baseline and follow-up.

Findings

The following outcomes were observed across the key measures:

1. Child behaviour problems – Carer completed Strengths and Difficulties Questionnaire (SDQ)

SDQs completed by carers in KEEP Standard showed significant improvements from baseline to follow-up on the sub-scale scores of emotional distress and prosocial behaviours compared with the control group. No statistically significant differences were found on the total SDQ score or on sub-scales relating to conduct problems, hyperactivity or peer problems.

2. Parenting skills and styles – Parenting Scale Improvements in parenting were observed for both the intervention and control groups; however, the intervention group showed significantly greater improvements compared with the control group at follow-up. This was seen in the total Parenting Scale score as well as on the verbosity and laxness sub-scales. Both the intervention and control groups improved from baseline to follow-up on the over-reactivity sub-scale, however, these improvements were not significantly different between the groups.

3. Carer well-being – Warwick-Edinburgh Mental Well-being Scale (WEMWBS)

Although the intervention group showed greater improvements in carer well-being compared with the control group, no statistically significant difference between the groups was found. Fewer carers completed the WEMWBS. This, coupled with the high degree of variation in carers’ WEMWBS scores, is likely to be contributing to the non-significant finding as it increases the size of the standard errors.

In order to provide a contextual understanding of the KEEP Standard training, 15 carers who participated in KEEP Standard were interviewed, with nine interviews taking place before the training and six after the training. Carers were very positive about the programme, and particularly praised the format of sharing strategies and experiences with other carers. Carers felt that the programme improved how they responded to challenging behaviour and equipped them with new strategies. Most carers described changes in their child’s behaviour as a consequence.

Although the sample of carers and data on placement stability was too small to provide a statistical analysis of child outcomes and associated cost savings, loose inferences from qualitative interviews suggested improvements in placement stability and the potential avoidance of costs resulting from carers seeking additional support to manage children’s challenging behaviours.