Does a research article's country of origin affect perception of its quality and relevance? A national trial of US public health researchers

Objectives The source of research may influence one's interpretation of it in either negative or positive ways, however, there are no robust experiments to determine how source impacts on one's judgment of the research article. We determine the impact of source on respondents’ assessment of the quality and relevance of selected research abstracts. Design Web-based survey design using four healthcare research abstracts previously published and included in Cochrane Reviews. Setting All Council on the Education of Public Health-accredited Schools and Programmes of Public Health in the USA. Participants 899 core faculty members (full, associate and assistant professors) Intervention Each of the four abstracts appeared with a high-income source half of the time, and low-income source half of the time. Participants each reviewed the same four abstracts, but were randomly allocated to receive two abstracts with high-income source, and two abstracts with low-income source, allowing for within-abstract comparison of quality and relevance Primary outcome measures Within-abstract comparison of participants’ rating scores on two measures—strength of the evidence, and likelihood of referral to a peer (1–10 rating scale). OR was calculated using a generalised ordered logit model adjusting for sociodemographic covariates. Results Participants who received high income country source abstracts were equal in all known characteristics to the participants who received the abstracts with low income country sources. For one of the four abstracts (a randomised, controlled trial of a pharmaceutical intervention), likelihood of referral to a peer was greater if the source was a high income country (OR 1.28, 1.02 to 1.62, p<0.05). Conclusions All things being equal, in one of the four abstracts, the respondents were influenced by a high-income source in their rating of research abstracts. More research may be needed to explore how the origin of a research article may lead to stereotype activation and application in research evaluation.

[1]  Role of a research ethics committee in follow-up and publication of results , 2003, The Lancet.

[2]  J. Walley,et al.  Research into practice: 10 years of international public health partnership between the UK and Swaziland. , 2010, Journal of public health.

[3]  J. Lowery,et al.  Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science , 2009, Implementation science : IS.

[4]  T. Opthof,et al.  The significance of the peer review process against the background of bias: priority ratings of reviewers and editors and the prediction of citation, the role of geographical bias. , 2002, Cardiovascular research.

[5]  S. Huttly,et al.  Effect of community-based peer counsellors on exclusive breastfeeding practices in Dhaka, Bangladesh: a randomised controlled trial , 2000, The Lancet.

[6]  Behnam Shakiba,et al.  BMC Medical Research Methodology BioMed Central Correspondence , 2006 .

[7]  A. Tversky,et al.  The framing of decisions and the psychology of choice. , 1981, Science.

[8]  D. Baldwin,et al.  The future of antidepressant pharmacotherapy. , 2003, World psychiatry : official journal of the World Psychiatric Association.

[9]  N. Dogra,et al.  Learning from low income countries: what are the lessons?: Partnerships in mental health are possible without multidisciplinary teams , 2004, British medical journal.

[10]  W. Bilkey,et al.  Country-of-Origin Effects on Product Evaluations , 1982 .

[11]  Russel L. Thompson,et al.  A Meta-Analysis of Response Rates in Web- or Internet-Based Surveys , 2000 .

[12]  James M Provenzale,et al.  Peer review at the American Journal of Roentgenology: how reviewer and manuscript characteristics affected editorial decisions on 196 major papers. , 2004, AJR. American journal of roentgenology.

[13]  Matthew J. Harris.  Research papers should omit their authors’ affiliations , 2014, BMJ : British Medical Journal.

[14]  J. Sterne.  Essentials of Medical Statistics , 1991 .

[15]  Robert E Black,et al.  Effect of community-based behaviour change management on neonatal mortality in Shivgarh, Uttar Pradesh, India: a cluster-randomised controlled trial , 2008, The Lancet.

[16]  E. Susser,et al.  Topics for our times: can we learn from the care of persons with mental illness in developing countries? , 1996, American journal of public health.

[17]  A. Rosen.  Destigmatizing day-to-day practices: what developed countries can learn from developing countries. , 2006, World psychiatry : official journal of the World Psychiatric Association.

[18]  J. Bargh,et al.  Automaticity of social behavior: direct effects of trait construct and stereotype-activation on action. , 1996, Journal of personality and social psychology.

[19]  T. Kaptchuk,et al.  Effect of interpretive bias on research evidence. , 2003, BMJ.

[20]  G. Guyatt,et al.  GRADE guidelines 6. Rating the quality of evidence--imprecision. , 2011, Journal of clinical epidemiology.

[21]  Gordon H Guyatt,et al.  GRADE guidelines: 5. Rating the quality of evidence--publication bias. , 2011, Journal of clinical epidemiology.

[22]  D. Yach,et al.  Assessment of epidemiological and HIV/AIDS publications in Africa. , 1992, International journal of epidemiology.

[23]  Jeroen J. Bax,et al.  The wealth of nations and the dissemination of cardiovascular research. , 2013, International journal of cardiology.

[24]  V. Patel,et al.  International representation in psychiatric literature , 2001, British Journal of Psychiatry.

[25]  L. Ross,et al.  Biased Assimilation and Attitude Polarization: The Effects of Prior Theories on Subsequently Considered Evidence , 1979 .

[26]  Suresh Kumar.  Learning from low income countries: what are the lessons?: Palliative care can be delivered through neighbourhood networks , 2004, BMJ : British Medical Journal.

[27]  O. Tutarel.  Composition of the editorial boards of leading medical education journals , 2004, BMC medical research methodology.

[28]  D. Yach,et al.  Research gap in cardiovascular disease in developing countries , 2003, The Lancet.

[29]  R. Peterson,et al.  A Meta-Analysis of Country-of-Origin Effects , 1995 .

[30]  A. Link.  US and non-US submissions: an analysis of reviewer bias. , 1998, JAMA.

[31]  R. MacCoun,et al.  Biases in the interpretation and use of research results. , 1998, Annual review of psychology.

[32]  R. Horton.  North and South: bridging the information gap , 2000, The Lancet.

[33]  A. Malmivaara.  Methodological considerations of the GRADE method , 2014, Annals of medicine.

[34]  A. Wakefield,et al.  Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. , 1998, Lancet.

[35]  Fiona Godlee,et al.  Eighth international congress on peer review in biomedical publication , 2015, BMJ : British Medical Journal.

[36]  G. Robert,et al.  Diffusion of innovations in service organizations: systematic review and recommendations. , 2004, The Milbank quarterly.

[37]  WPA and disaster response: new policies and actions. , 2006, World psychiatry : official journal of the World Psychiatric Association.

[38]  D. Singh.  Publication bias - a reason for the decreased research output in developing countries , 2006 .

[39]  S. Syed,et al.  Reverse innovation in global health systems: towards global innovation flow , 2013, Globalization and Health.

[40]  D. Baguley,et al.  International health links: an evaluation of partnerships between health-care organizations in the UK and developing countries , 2006, Tropical doctor.

[41]  Diana Silver,et al.  ‘They hear “Africa” and they think that there can’t be any good services’ – perceived context in cross-national learning: a qualitative study of the barriers to Reverse Innovation , 2015, Globalization and Health.

[42]  James W. Dearing,et al.  Portraying the New: Communication Between University Innovators and Potential Users , 1994 .

[43]  S. Ceci,et al.  Peer-review practices of psychological journals: The fate of published articles, submitted again , 1982, Behavioral and Brain Sciences.

[44]  Gerald Gartlehner,et al.  [GRADE guidelines: 8. Rating the quality of evidence - indirectness]. , 2012, Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen.

[45]  E. Ferlie,et al.  The Nonspread of Innovations: the Mediating Role of Professionals , 2005 .

[46]  Howard Balshem,et al.  GRADE guidelines: 3. Rating the quality of evidence. , 2011, Journal of clinical epidemiology.

[47]  D. Berwick.  Lessons from developing nations on improving health care , 2004, BMJ : British Medical Journal.

[48]  Gordon H Guyatt,et al.  GRADE guidelines: 9. Rating up the quality of evidence. , 2011, Journal of clinical epidemiology.

[49]  V. Patel,et al.  Learning from low income countries: mental health , 2004, BMJ : British Medical Journal.

[50]  C. Lam.  Knowledge can flow from developing to developed countries , 2000, BMJ : British Medical Journal.

[51]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[52]  Gerald Gartlehner,et al.  [GRADE guidelines: 7. Rating the quality of evidence - inconsistency]. , 2012, Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen.

[53]  G. Guyatt,et al.  GRADE guidelines: 4. Rating the quality of evidence--study limitations (risk of bias). , 2011, Journal of clinical epidemiology.

[54]  H. Swartz,et al.  Managing the global burden of depression: lessons from the developing world. , 2003, World psychiatry : official journal of the World Psychiatric Association.

[55]  Marcel Tanner,et al.  Representation of authors and editors from countries with different human development indexes in the leading literature on tropical medicine: survey of current evidence , 2004, BMJ : British Medical Journal.

[56]  E. Ferlie,et al.  Interlocking Interactions, the Diffusion of Innovations in Health Care , 2002 .

[57]  A. Haines,et al.  Achieving child survival goals: potential contribution of community health workers , 2007, The Lancet.

[58]  Lord Nigel Crisp.  Turning the world upside down , 2010 .

[59]  A R Jadad,et al.  Assessing the quality of reports of randomized clinical trials: is blinding necessary? , 1996, Controlled clinical trials.