Interrater and intrarater agreement on heart murmurs

Abstract Objective To investigate interrater and intrarater agreement between physicians and medical students on heart sound classification from audio recordings, and factors predicting agreement with a reference classification. Design Intra- and interrater agreement study. Subjects Seventeen GPs and eight cardiologists from Norway and the Netherlands, eight medical students from Norway. Main outcome measures Proportion of agreement and kappa coefficients for intrarater agreement and agreement with a reference classification. Results The proportion of intrarater agreement on the presence of any murmur was 83% on average, with a median kappa of 0.64 (range k = 0.09–0.86) for all raters, and 0.65, 0.69, and 0.61 for GPs, cardiologist, and medical students, respectively. The proportion of agreement with the reference on any murmur was 81% on average, with a median kappa of 0.67 (range 0.29–0.90) for all raters, and 0.65, 0.69, and 0.51 for GPs, cardiologists, and medical students, respectively. Distinct murmur, more than five years of clinical practice, and cardiology specialty were most strongly associated with the agreement, with ORs of 2.41 (95% CI 1.63–3.58), 2.19 (1.58–3.04), and 2.53 (1.46–4.41), respectively. Conclusion We observed fair but variable agreement with a reference on heart murmurs, and physician experience and specialty, as well as murmur intensity, were the factors most strongly associated with agreement. Key points: Heart auscultation is the main physical examination of the heart, but we lack knowledge of inter- and intrarater agreement on heart sounds. • Physicians identified heart murmurs from heart sound recordings fairly reliably compared with a reference classification, and with fair intrarater agreement. • Both intrarater agreement and agreement with the reference showed considerable variation between doctors • Murmur intensity, more than five years in clinical practice, and cardiology specialty were most strongly linked to agreement with the reference.

[1]  B. Prendergast,et al.  2021 ESC/EACTS Guidelines for the management of valvular heart disease. , 2021, European heart journal.

[2]  B. Prendergast,et al.  Global epidemiology of valvular heart disease , 2021, Nature Reviews Cardiology.

[3]  Sathish Kumar Jayapal,et al.  Global Burden of Cardiovascular Diseases and Risk Factors, 1990–2019 , 2020, Journal of the American College of Cardiology.

[4]  Achim Zeileis,et al.  Various versatile variances : An object-oriented implementation of clustered covariances in R Working , 2017 .

[5]  S. Vanbelle,et al.  The use of spectrograms improves the classification of wheezes and crackles in an educational setting , 2020, Scientific Reports.

[6]  James D. Thomas,et al.  A Deep Learning Algorithm for Automated Cardiac Murmur Detection Via a Digital Stethoscope Platform , 2020, medRxiv.

[7]  S. Stewart,et al.  Poor Long-term Survival in Patients with Moderate Aortic Stenosis. , 2019, Journal of the American College of Cardiology.

[8]  K. Ahn,et al.  Radiographic prediction of lunate morphology in Asians using plain radiographic and capitate-triquetrum distance analyses: reliability and compatibility with magnetic resonance arthrography (MRA) findings , 2019, BMC Musculoskeletal Disorders.

[9]  B. Prendergast,et al.  Cardiac auscultation poorly predicts the presence of valvular heart disease in asymptomatic primary care patients , 2018, Heart.

[10]  C. Otto.  Mind the gap: missed valve disease diagnosis , 2018, Heart.

[11]  H. Schirmer,et al.  The evolving epidemiology of valvular aortic stenosis. The Tromsø Study , 2012, Heart.

[12]  B Bridgewater,et al.  Valvular heart disease: the next cardiac epidemic , 2010, Heart.

[13]  A. Hrõbjartsson,et al.  Effect of teaching and type of stethoscope on cardiac auscultatory performance. , 2006, American heart journal.

[14]  Jasminka M Vukanovic-Criley,et al.  Competency in cardiac examination skills in medical students, trainees, physicians, and faculty: a multicenter study. , 2006, Archives of internal medicine.

[15]  A. Zeileis.  Econometric Computing with HC and HAC Covariance Matrix Estimators , 2004 .

[16]  Per Hasvold,et al.  [May heart murmurs be assessed by telemedicine?]. , 2003, Tidsskrift for den Norske laegeforening : tidsskrift for praktisk medicin, ny raekke.

[17]  L. Nieman,et al.  The Teaching and Practice of Cardiac Auscultation during Internal Medicine and Cardiology Training: A Nationwide Survey , 1993, Annals of Internal Medicine.

[18]  J. Munro.  Macleod's Clinical Examination , 1990 .

[19]  S. Abraham,et al.  A STUDY OF PHYSICIAN VARIATION IN HEART-SOUND INTERPRETATION. , 1964, The Medical annals of the District of Columbia.

[20]  S. A. Levine.  AUSCULTATION OF THE HEART* , 1948, The Heart bulletin.

[21]  Achim Zeileis,et al.  Diagnostic Checking in Regression Relationships , 2015 .

[22]  A. Hrõbjartsson,et al.  Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. , 2011, Journal of clinical epidemiology.

[23]  S. Helgerson.  Clinical Epidemiology: How to Do Clinical Practice Research , 2006 .

[24]  K. Ho,et al.  Factors influencing cardiac auscultation proficiency in physician trainees. , 2005, Singapore medical journal.