Evaluating family hubs

Evaluation is not something that happens only at the end, and nor is it something that happens independently. It is important that evidence and evaluation are inclusive and that families and the workforce are engaged in the process at an early stage. It is also essential that all staff understand the purpose of the evaluation and understand the part they play in collecting data to feed into the evaluation.

Families should be just as involved in the evaluation process as they are in wider service design and commissioning. This will help ensure that the approach adopted is successful and the outcomes measured are relevant and meaningful to families. For more information on working with families, please read our participation, service design and commissioning module.

Different stakeholders will have different skills and perspectives to offer when considering how information should be collected, analysed and interpreted. For example, some professionals may have strong technical understanding of the data, while others may bring a strategic or expert perspective. Front line staff are likely to be the ones collecting data about outcomes for families. Lack of understanding of evaluation can sometimes result in mistrust of data collection and a fear that workers are being assessed on numbers that do not reflect their performance or the progress they have made with families. This is why it is critical to engage all staff early in evaluation planning and offer regular training on the purpose of evaluation and the reasons for data collection that can not only help to establish trust but also result in better data collection and therefore better data quality. You can read more about cultivating trust in our integration and relational practice module.

In addition to engaging all staff in the evaluation planning process, it can also be useful to engage staff in the interpretation of findings. This is particularly necessary for data that is largely quantitative and so may be scrutinised as not telling the whole story. Parent, carer and young person representatives should be included in conversations around interpretation of outcome data to help complete the picture from a family perspective. Ensuring collaboration with families and frontline staff in data analysis will not only improve the validity of the evaluation but reinforces the broader focus of evaluation as a tool to promote the best outcomes for families, and not an indicator of individual performance or just a way to placate funders.

There can be value in employing an external evaluator to undertake evaluation of your family hubs offer as it ensures that interpretation of findings is impartial. However, for many local authorities who are not involved in a national evaluation, the prospect of employing an external evaluator is not financially viable. In these instances, it is possible to undertake a successful evaluation internally by analysing the data collected in line with the desired outcomes for families.  

Important elements to consider if undertaking an internal evaluation of the family hubs offer include: 

1. examining the interaction between family hubs and the local context (including what other offers are available and local population needs assessments). 
2. referencing back to the theory of change behind the family hub offer, including any assumptions made. 
3. ensuring that all stakeholders are included in the evaluation and diverse perspectives are considered. 
4. considering uncertainties and things not yet known. 
5. looking at how the family hubs offer can be improved. 

Some local authorities who are eligible to receive transformation funding for their family hub development may be selected to participate in a national evaluation of the programme.  

The national evaluation programmes outlined below are conducted by external evaluation specialists and academics. 

A mixed-methods evaluation of a subgroup of TF2-funded local authorities to assess the development and impact of all aspects of the family hubs and Start for Life programme.

A mixed-methods evaluation to document the family hubs transformation journey across all 12 TF1-funded local authorities.

An in-depth evaluation of the Start for Life strand of the transformation programme. This evaluation will focus on the development and impact of the Start for Life offer amongst a subgroup of TF2 funded local authorities.   

The Ecorys partnership is a collaboration between researchers from Ecorys UK, with Clarissa White Research and Starks Consulting Ltd. The evaluation comprised of a bespoke individual evaluation of family hubs in five local authorities and a ‘realist synthesis’ combining insights across all five evaluations using both quantitative and qualitative data from staff and families. The five local authorities involved in this evaluation are:

1. Bristol City Council
2. Essex County Council
3. Leeds City Council
4. Sefton Council
5. Suffolk County Council.

The evaluation runs over 24 months. The interim report can be found here.

This evaluation focuses on the family hub model in Doncaster Metropolitan Borough Council (DMBC), chosen as one of the most well-developed locality-based family hub models in England. The evaluation involves assessment of implementation and performance, an outcomes and impact evaluation and a cost-effectiveness analysis.

The evaluation runs over 24 months and you can see the interim reporting

The final results will be published in late summer 2023.

The next section provides guidance on how to evaluate if you are not able to commission independent evaluators.

People sometimes think about evaluation as something to do at the end of an intervention or system change, but evaluation is most useful when embedded from the beginning. The overall outcomes of the offer are likely to be actively considered during the planning stages and with these outcomes in mind it is easier to think about how to measure progress. We suggest this forms part of your early discussions as part of the family hub development process when developing a Theory of Change model for your local area. 

By thinking about evaluation during the planning stages, it enables you to collect data at baseline, from before transformation to family hubs begins, as well as throughout the transformation journey. Collecting baseline data enables the opportunity for comparison between the beginning and at different stages of programme transformation to understand how things have changed and what the trajectory of that change looks like for services and for families. Collecting data throughout makes it possible to monitor across the lifespan and make changes, if necessary, in real time.

Analysing outcome data, regardless of whether it is quantitative or qualitative in nature, requires some degree of subjective interpretation. It is impossible to measure all the potential benefits a family may experience from a service or an intervention, and the metrics we use to measure outcomes will, at best, provide only a glimpse into the actual experience for that family. A more holistic approach is to combine the data collected from routine questionnaires with asking families questions about their experience and the impact they feel it has had on them with qualitative stories and feedback from discussions with families about their experience of support. Focusing on the meaningful impact for families can be useful for data analysts to establish ‘indicators’ for broader outcomes.

Although not every local authority will be able to work with independent evaluators, it is useful to consider their role because independent evaluation can bring a level of objectivity and rigour that self-evaluation may lack.

Services working together towards shared outcomes is a key feature of the integrated family hub approach. The family hub partnership should share a local theory of change and a population-level or cohort-level outcomes framework, with measurement of family-level outcomes feeding into local population-level outcomes, and commitment among local partners to develop this outcomes framework further. 

Although you will be basing family-level outcomes on an overarching outcomes framework, you may need to use additional family–level metrics at points to meet the needs of your local population. Clear planning will be crucial here to avoid inconsistencies in how you define, measure and talk about the experiences of children and families and the outcomes you intend to achieve as outlined in your theory of change model. 

Partners within your family hub may already have metrics that they use to measure the effectiveness of their service, and these will likely be linked to their individual service-level key performance indicators. During the planning stages for your family hub transformation, it can be useful to bring all of these to the table together and determine whether the existing metrics fit with the outcomes set out for your community in your theory of change model. This process will also help to identify any gaps you may have in your data collection. 

Here are some things to consider when selecting tools to measure your family-level outcomes: 

1. validity: how accurately does this tool measure the outcome?   
2. reliability: how consistent is the measure in relation to your outcomes?  
3. accessibility: how accessible is the tool to the children, young people or parents and carers it is intended for?  
4. acceptability: will this tool be well received by families?  
5. inclusivity: is this tool applicable across cultures and has it been validated to be used across cultures? 

Coproduction is key when considering which family-level metrics would best reflect the needs of the families accessing your family hub.  

To help you navigate the many outcome tools that complement the supporting families framework, please see this set of documents produced by Kindred2 in  collaboration with the For Baby’s Sake Trust and EIF.  

It is crucial that you consider equity, diversity and inclusion. Many frameworks refer to child characteristics or contextual factors such as gender and ethnicity, special educational needs and disabilities (SEND), and economic context, which is often gauged using rates of parental unemployment, take-up of benefits such as universal credit, or children’s eligibility for free school meals. Often times, these metrics are ‘deficit-focused’ rather than ‘strengths-based’, and this can be problematic when using an outcomes-led tool to motivate the wider workforce. Consider the ‘tone’ of the data collected and the message you wish to convey with this information, ensuring there is a balance between strengths-focus and deficit-focus. 

Goal based outcomes (GBOs) can be used with children and young people with learning disabilities and their parents, carers and networks. You can find out more about using GBOs with children and young people with learning disabilities and how this can increase their power, agency and voice in the CORC resource Goals and goal based outcomes. 

Population-level outcomes are outcomes that can be measured and compared across a large group of people – a population. Because in many cases they draw on national statistics and data is available publicly, comparisons can be made with populations in other local authorities or regions, or nationally. This makes population-level outcomes useful tools for understanding the ‘big picture’. They differ from individual-level data (for example, data about the family, young person, child or carer), which can be used to measure individual responses to specific interventions or services.  

Nationally available metrics are usually only published annually, and you will need to account for this in planning. Examples of key public data sources include:  

1. Public Health England (PHE) – Fingertips database  
2. Office for National Statistics, for example: young people’s wellbeing measures, personal wellbeing (Annual Population Survey), Crime Survey for England and Wales Crime Survey for England and Wales The Office of the Children’s Commissioner’s local vulnerability profile 
3. DfE’s Early Years Outcome Dashboard and local authority interactive tool (LAIT). 

Interpreting national datasets at a local level requires some degree of caution as geographical borders of LAs are not always co-terminus with public health ‘footprints’. You will need to contextualise population service-level outcomes and choose appropriate metrics to do so. Have a look at the outcomes and datasets out there already, some of which are included below: 

PHE’s Outcome Framework 2019-22, which aims to improve and protect the nation’s health and wellbeing and improve the health of the poorest fastest using the following framework: 

1. improving the wider determinants of health – improvements against wider factors which affect health and wellbeing and health inequalities.  
2. heath improvement – people are helped to live healthy lifestyles, make healthy choices and health inequalities lessen as a result. 
3. health protection – the population’s health is protected from major incidents and other threats, while health inequalities lessen.

DfE’s early years foundation stage highlights three prime areas of learning and development that are important for building a foundation for children to thrive: communication and language, physical development and personal, social and emotional development. The framework cites four specific areas through which this foundation is strengthened and applied: literacy, mathematics, understanding the world and expressive arts and design.   

1. Family hubs are delivering evidence-based programmes and interventions with a commitment to increase this across more of their services. 
2. Local strategic needs assessments include data on family needs. 
3. Family feedback data collected and collated on experiences of using family hub services. 
4. Regular family hub network staff and professional time for reflective practice and learning from past experience and projects. 

1. Regular reviews of the latest evidence base on family hub practice, programme and intervention effectiveness 
2. Regular family hub network staff-training and learning and development on delivering evidence-based programmes and interventions. 
3. Local evaluation evidence for family hubs and their constituent services is regularly reviewed at operational, management and strategic level and leads to improvements and refinement of practice, services and interventions. 
4. Regular events, forums and supervision time for professionals and staff to reflect on practice and learn from projects and pieces of work as part of the family hub network. 

1. Evidence-based programmes and interventions are at the core of family hub service provision and are delivered with fidelity across most services. 
2. Robust and up-to-date multi-agency data (for example health, education, social care) on families is routinely analysed, covering population needs and service use, based on data from across the family hub network. The analysis is routinely used (as it pertains to family hubs) to identify target groups, design services, agree priorities, forecast trends and plan, set strategy, and influence wider family and community strategies. 
3. Routine monitoring, tracking and analysing of family hub service performance using valid and reliable outcome metrics, and linking with caseload data, children social care data, and data from local and national partners. Proven effectiveness of family hub services at improving child and family outcomes, with findings published. 
4. Established evaluation partners that offer independent scrutiny and review of the family hub network. 
5. Regular benchmarking, learning and activities that assure the quality of the services against intended outcomes, alongside service users experiences. Activities may be undertaken with other local authorities with family hubs and could include data and outcome benchmarking or themed audits.