Over the past eight years, SAFE Ireland, together with international partners Michigan State University and Scottish Women’s Aid, has taken a lead role in developing and implementing outcome evaluation measures within frontline domestic violence in Ireland. Most recently SAFE Ireland published Safety and Change (2008), a national refuges’ outcome evaluation research study and the Outcome Evaluation of Domestic Violence Support and Information Services Pilot (2010). The final research study in this process has been the national implementation of outcome evaluation measures across member support, information and advocacy services in Ireland. The national implementation was undertaken in parallel with a similar process with Scottish Women’s Aid. This report is due to be published in the near future.
Why outcome evaluation?
SAFE Ireland considers that outcome evaluation offers a practical, manageable and effective method for domestic violence service providers to understand and improve the impact of their services on women and children.
Outcome evaluation is an assessment and evaluation tool that measures benefits, impacts or changes that occur as a direct result of interventions.
In relation to domestic violence, outcome evaluation can be used to assess service provision effectiveness by identifying changes in women’s knowledge, attitudes, skills, behaviour, expectations, emotional status or life circumstances due to the service provided to them (Sullivan 1998). If outcomes are specific, measurable and directly tied to programme activities then their use can provide direct feedback to service providers and stakeholders on the effectiveness of the intervention.
Outcome Evaluation of Domestic Violence Support and Information Services in Ireland – A Pilot Study 2009Download this report here >> (PDF 2.4Mb)
Safety and Change – A national study of support needs and outcomes for women accessing refuge provision in IrelandDownload this report here >> (PDF 2.2Mb)
SAFE Ireland has developed strong research and partnership links with two organisations in relation to outcome evaluation, Michigan State University and Scottish Women’s Aid. This research project was run in conjunction with Scottish Women’s Aid, who completed similar national research with its member groups in a parallel process.(See Scottish Women’s Aid Report (PDF)) It was felt that undertaking the work on a transnational basis would offer the opportunity to share expertise, rigorously debate all elements of the project and share findings, thus adding to the knowledge base in relation to women’s needs and outcomes. The results of the joint work is considered in a further report (Forthcoming). Professor Cris Sullivan from Michigan State University acted as advisor to the project.
SAFE Ireland Development and Implementation of Outcome Evaluation in Ireland 2003-present
- Research report: Social Service or Social Change: (2003) highlighted the importance for frontline domestic violence services of engaging in outcome evaluation.
- Shelters@net: SAFE Ireland participated in the DAPHNE-funded Shelters@net project, a comparative study on refuge provision, involving France, Denmark, Portugal, Scotland, Slovenia and Ireland. Findings highlighted that there were no national or regional structures in existence for evaluation, despite an awareness of its importance.
Domestic Violence services highlighted two main barriers to implementing outcome evaluation measures; the absence of agreed and effective outcome evaluation tools and systems, and a lack of resources to develop the tools and implement them.
- Refuge Evaluation Modeling: As a step towards addressing this gap, further DAPHNE funding was secured for a follow-up project. With Scottish Women’s Aid and CESIS, (an independent social research organisation in Portugal) we undertook a transnational project to create an outcome evaluation model for evaluating the effectiveness of women’s refuges and to pilot this model in women’s refuges across the three countries. The project, known as the Refuge Evaluation Modeling (REM) project benefited significantly from the support of Professor Cris Sullivan, Michigan State University, a highly experienced evaluator of domestic violence programmes in North America. The process and developments of this project are presented in its final report, (CESIS et al 2007), as well as in a peer-reviewed research article (Sullivan et al, 2008).
- A National Outcome Evaluation Study of Refuges in Ireland 2008-2009
- (18 out of 19 refuges participated)
- 515 responses (58% of women arriving in refuge and 39% of women leaving refuge)
- Barriers to participation
- Resources of organisation
- Literacy issues
- Crisis nature of work
- Implementation of recommendations of evaluation (2009 & ongoing)
- Individual Reports fed back to each refuge with more than 15 of each form
- Feedback to individual refuges re specific recommendations from women
- A pilot study of the support, information and outreach components of domestic violence services (2009-2010)
- A national study of the support, information and outreach components of domestic violence services (2010-2011 )
- Currently writing up national report (414 forms across 25 services)
- Individual data reports to be compiled and returned to all participating organsiations with more than 15 responses
- Partnership with Scottish Women’s Aid who have rolled out parallel study with Domestic Violence Services in Scotland in 2010
- Co writing detailed methodology based on implementation and learning
- Co writing comparative report on two countries data sets (802 responses)
- Designing qualitative study with Michigan State University (Prof Sullivan) in order to research the detail of the interventions that women received linked to their needs and outcomes
- Updating Scottish Women’s Aid toolkit to translate it into Irish context and disseminate to Irish domestic violence services for ongoing implementation of Outcome Evaluation tools.
Morton S (2003). Social Service or Social Change? Issues and Challenges for Frontline Domestic Violence Services. Athlone: NNWRSS.
NNWRSS (2008). Outcome Evaluation in Irish Refuges: Development and Piloting of Outcome Evaluation Model in Irish Refuges as part of the Daphne REM Project. Athlone: NNWRSS.
Riger S, Bennet L, Wasco S, Schewe P, Frohmann L, Camcacho J & Campbell R (2002). Evaluating Services for Survivors of Domestic Violence and Sexual Assault. California: Sage.
SAFE Ireland (2009) Safety and Change: A national study of support needs and outcomes for women accessing refuge provision in Ireland. Safe Ireland, Athlone.
SAFE Ireland (2009) Outcome Evaluation of Domestic Violence Support and Information Services – A Pilot Study SAFE Ireland, Athlone.
Sullivan C (2001). Evaluating the Outcomes of Domestic Violence Service Programs: Some Practical Considerations and Strategies. Applied Research Forum National Electronic Network on Violence Against Women.
Sullivan, C (1998). Outcome evaluation strategies for domestic violence programs: A practical guide. Harrisburg: Pennsylvania Domestic Violence Coalition.
Sullivan C, Baptista I, O’Halloran S, Okroj L, Morton S & Sutherland Steward C (2008). Evaluating the Effectiveness of Women’s Refuges: A Multi-Country Approach to Model Development. International Journal of Comparative and Applied Criminal Justice, 2008 Volume 32 (2) 291-308.[/accordion]