Assessing research impact: Australian Research Council criteria and the case of Family Wellbeing research.

Researchers worldwide are increasingly reporting the societal impact of their research as part of national research productivity assessments. However, the challenges they encounter in developing their impact case studies against specified government assessment criteria and how pitfalls can be mitigated are not reported. This paper examines the key steps taken to develop an Aboriginal Family Wellbeing (FWB) empowerment research impact case study in the context of an Australian Research Council (ARC) pilot research impact assessment exercise and the challenges involved in applying the ARC criteria. The requirement that researchers demonstrate how their institutions support them to conduct impactful research has the potential to create supportive environments for researchers to be more responsive to the needs of users outside academia. However, the 15-year reference period for the associated research underpinning the reported impact and the focus on researcher's current institutional affiliation constitute potential constraints to demonstrating the true impact of research. For researchers working with Indigenous people, relationships that build over long periods of time, irrespective of university affiliation, are critical to conducting impactful research. A more open-ended time-frame, with no institutional restrictions for the 'associated research' provides the best opportunity to demonstrate the true benefits of research not only for Indigenous people but for Australian society more broadly.

[1]  N. Jarman,et al.  Beyond the Academy: Applying Anthropological Research, A Case Study of Demonstrating Impact in the U.K. 2014 REF , 2015 .

[2]  A. Kothari,et al.  Research impact of systems-level long-term care research: a multiple case study , 2017, Health Research Policy and Systems.

[3]  Komla Tsey,et al.  The characteristics and reporting quality of research impact case studies: A systematic review. , 2019, Evaluation and program planning.

[4]  Megan Williams,et al.  Learning about Aboriginal health and wellbeing at the postgraduate level: novel application of the Growth and Empowerment Measure. , 2019, Rural and remote health.

[5]  I. Kinchin,et al.  Does Indigenous health research have impact? A systematic review of reviews , 2017, International Journal for Equity in Health.

[6]  C. Donovan,et al.  Evaluation of the impact of National Breast Cancer Foundation‐funded research , 2014, The Medical journal of Australia.

[7]  E. Woodward,et al.  Transforming Cross-Cultural Water Research through Trust, Participation and Place , 2016 .

[8]  James Raftery,et al.  Models and applications for measuring the impact of health research: update of a systematic review for the Health Technology Assessment programme. , 2016, Health technology assessment.

[9]  Barbara Befani,et al.  Process Tracing and Bayesian Updating for impact evaluation , 2017 .

[10]  M. Buxton,et al.  Payback arising from research funding: evaluation of the Arthritis Research Campaign. , 2005, Rheumatology.

[11]  S. Hanney,et al.  Four centuries on from Bacon: progress in building health research systems to improve health systems? , 2014, Health Research Policy and Systems.

[12]  Ousmane Coulibaly,et al.  The impact of agricultural research on productivity and poverty in sub-Saharan Africa , 2009 .

[13]  P. McGrath,et al.  Examining the impact of health research facilitated by small peer-reviewed research operating grants in a women's and children's health centre , 2010, BMC Research Notes.

[14]  S. Ryan,et al.  McUniversities revisited: a comparison of university and McDonald's casual employee experiences in Australia , 2015 .

[15]  Y. Cadet‐James,et al.  Adapting the ‘Family Wellbeing’ empowerment program to the needs of remote Indigenous school children , 2005, Australian and New Zealand journal of public health.

[16]  Janya McCalman.  A grounded theory of program transfer: how an Aboriginal empowerment initiative became "bigger than a program" , 2012 .

[17]  Ilona Papajcsik.  Strategies and practices for promoting the social and emotional wellbeing of Aboriginal and Torres Strait Islander people , 2013 .

[18]  K. Tsey,et al.  Evaluating Aboriginal empowerment programs: the case of Family WellBeing , 2000, Australian and New Zealand journal of public health.

[19]  G. D. Holt,et al.  Enablers, challenges and relationships between research impact and theory generation , 2016 .

[20]  I. Kinchin,et al.  No one’s discussing the elephant in the room: contemplating questions of research impact and benefit in Aboriginal and Torres Strait Islander Australian health research , 2015, BMC Public Health.

[21]  J. Raftery,et al.  Assessing the impact of England's National Health Service R&D Health Technology Assessment program using the “payback” approach , 2009, International Journal of Technology Assessment in Health Care.

[22]  T. Greenhalgh,et al.  Research impact in the community-based health sciences: an analysis of 162 case studies from the 2014 UK Research Excellence Framework , 2015, BMC Medicine.

[23]  Y. Cadet‐James,et al.  Empowerment-Based Research Methods: A 10-Year Approach to Enhancing Indigenous Social and Emotional Wellbeing , 2007, Australasian psychiatry : bulletin of Royal Australian and New Zealand College of Psychiatrists.

[24]  Lynne McPherson,et al.  Connecting and Strengthening Young Aboriginal Men: A Family Wellbeing Pilot Study , 2016 .

[25]  B. McCormack.  Engaged scholarship and research impact: integrating the doing and using of research in practice , 2011 .

[26]  I. Kinchin,et al.  Evaluating Research Impact: The Development of a Research for Impact Tool , 2016, Front. Public Health.

[27]  K. Tsey,et al.  The Aboriginal Australian Family Wellbeing Program: A Historical Analysis of the Conditions That Enabled Its Spread , 2018, Front. Public Health.

[28]  P. Home,et al.  Impact of the UKPDS—an overview , 2008, Diabetic medicine : a journal of the British Diabetic Association.

[29]  Shyanika W. Rose,et al.  Scientific and Public Health Impacts of the NIEHS Extramural Asthma Research Program - Insights from Primary Data. , 2009, Research Evaluation.

[30]  H. Klieve,et al.  The evidence needed to demonstrate impact: A synthesis of the evidence from a phased social and emotional wellbeing intervention. , 2018, Evaluation and program planning.

[31]  Chris Doran,et al.  An approach to measuring and encouraging research translation and research impact , 2016, Health Research Policy and Systems.

[32]  K. Tsey,et al.  Acceptability of an Aboriginal Wellbeing Intervention for Supporters of People Using Methamphetamines , 2018, Australian Social Work.

[33]  R. Scoble,et al.  Assessment, evaluations, and definitions of research impact: A review , 2014 .

[34]  K. Tsey,et al.  Taking control of health: Gurriny's story of organisational change , 2010 .

[35]  A. Every,et al.  Evaluation of an Aboriginal empowerment program , 2000 .

[36]  Tom Walker,et al.  Impact assessment of policy-oriented international agricultural research: Evidence and insights from case studies , 2010 .

[37]  A. Bauman,et al.  Policy and practice impacts of applied research: a case study analysis of the New South Wales Health Promotion Demonstration Research Grants Scheme 2000–2006 , 2013, Health Research Policy and Systems.