Automated low‐contrast pattern recognition algorithm for magnetic resonance image quality assessment

Purpose: Low contrast (LC) detectability is a common test criterion for diagnostic radiologic quality control (QC) programs. Automation of this test is desirable in order to reduce human variability and to speed up analysis. However, automation is challenging due to the complexity of the human visual perception system and the ability to create algorithms that mimic this response. This paper describes the development and testing of an automated LC detection algorithm for use in the analysis of magnetic resonance (MR) images of the American College of Radiology (ACR) QC phantom. Methods: The detection algorithm includes fuzzy logic decision processes and various edge detection methods to quantify LC detectability. Algorithm performance was first evaluated using a single LC phantom MR image with the addition of incremental zero mean Gaussian noise resulting in a total of 200 images. A c‐statistic was calculated to determine the role of CNR to indicate when the algorithm would detect ten spokes. To evaluate inter‐rater agreement between experienced observers and the algorithm, a blinded observer study was performed on 196 LC phantom images acquired from nine clinical MR scanners. The nine scanners included two MR manufacturers and two field strengths (1.5 T, 3.0 T). Inter‐rater and algorithm‐rater agreement was quantified using Krippendorff's alpha. Results: For the Gaussian noise added data, CNR ranged from 0.519 to 11.7 with CNR being considered an excellent discriminator of algorithm performance (c‐statistic = 0.9777). Reviewer scoring of the clinical phantom data resulted in an inter‐rater agreement of 0.673 with the agreement between observers and algorithm equal to 0.652, both of which indicate significant agreement. Conclusions: This study demonstrates that the detection of LC test patterns for MR imaging QC programs can be successfully developed and that their response can model the human visual detection system of expert MR QC readers.

[1]  Atiba Omari Fitzpatrick.  Automated Quality Assurance for Magnetic Resonance Imaging with Extensions to Diffusion Tensor Imaging , 2005 .

[2]  M. McHugh.  Interrater reliability: the kappa statistic , 2012, Biochemia medica.

[3]  Xavier Robin,et al.  pROC: an open-source package for R and S+ to analyze and compare ROC curves , 2011, BMC Bioinformatics.

[4]  Jason Dowling,et al.  An open source automatic quality assurance (OSAQA) tool for the ACR MRI phantom , 2014, Australasian Physical & Engineering Sciences in Medicine.

[5]  R. Downey,et al.  Rating the ratings: Assessing the psychometric quality of rating data , 1980 .

[6]  D. Louis Collins,et al.  Automated Analysis of Multi Site MRI Phantom Data for the NIHPD Project , 2006, MICCAI.

[7]  J. Mendel.  Fuzzy logic systems for engineering: a tutorial , 1995, Proc. IEEE.

[8]  Graham Walker,et al.  Quality systems for medical imaging : guidelines for implementation and monitoring , 1999 .

[9]  Klaus Krippendorff,et al.  Computing Krippendorff's Alpha-Reliability , 2011 .

[10]  Klaus Krippendorff,et al.  Answering the Call for a Standard Reliability Measure for Coding Data , 2007 .

[11]  Francisco Herrera,et al.  Linguistic decision analysis: steps for solving decision problems under linguistic information , 2000, Fuzzy Sets Syst..

[12]  D. Dowsett,et al.  Physics of diagnostic imaging , 1998 .

[13]  Linda Shields,et al.  Content Analysis , 2015 .

[14]  M. Eliasziw,et al.  Statistical methodology for the concurrent assessment of interrater and intrarater reliability: using goniometric measurements as an example. , 1994, Physical therapy.

[15]  L. A. ZADEH,et al.  The concept of a linguistic variable and its application to approximate reasoning - I , 1975, Inf. Sci..

[16]  Jr-Yuan George Chiou,et al.  On replacing the manual measurement of ACR phantom images performed by MRI technologists with an automated measurement approach , 2016, Journal of magnetic resonance imaging : JMRI.

[17]  Ho-Ling Liu,et al.  Quality Assurance of Clinical MRI Scanners Using ACR MRI Phantom: Preliminary Results , 2004, Journal of Digital Imaging.

[18]  Mathias Davids,et al.  Fully-automated quality assurance in multi-center studies using MRI phantom measurements. , 2014, Magnetic resonance imaging.

[19]  Tzung-Pei Hong,et al.  Induction of fuzzy rules and membership functions from training examples , 1996, Fuzzy Sets Syst..

[20]  Jeffrey Weinreb,et al.  ACR MRI accreditation: yesterday, today, and tomorrow. , 2005, Journal of the American College of Radiology : JACR.

[21]  M. Pencina,et al.  Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation , 2004, Statistics in medicine.

[22]  A. G. Ramakrishnan,et al.  Quality assessment in magnetic resonance images. , 2010, Critical reviews in biomedical engineering.

[23]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[24]  Kevin A Hallgren,et al.  Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial. , 2012, Tutorials in quantitative methods for psychology.