The challenges of designing a benchmark strategy for bioinformatics pipelines in the identification of antimicrobial resistance determinants using next generation sequencing technologies

Next-Generation Sequencing (NGS) technologies are expected to play a crucial role in the surveillance of infectious diseases, with their unprecedented capabilities for the characterisation of genetic information underlying the virulence and antimicrobial resistance (AMR) properties of microorganisms.  In the implementation of any novel technology for regulatory purposes, important considerations such as harmonisation, validation and quality assurance need to be addressed.  NGS technologies pose unique challenges in these regards, in part due to their reliance on bioinformatics for the processing and proper interpretation of the data produced.  Well-designed benchmark resources are thus needed to evaluate, validate and ensure continued quality control over the bioinformatics component of the process.  This concept was explored as part of a workshop on "Next-generation sequencing technologies and antimicrobial resistance" held October 4-5 2017.   Challenges involved in the development of such a benchmark resource, with a specific focus on identifying the molecular determinants of AMR, were identified. For each of the challenges, sets of unsolved questions that will need to be tackled for them to be properly addressed were compiled. These take into consideration the requirement for monitoring of AMR bacteria in humans, animals, food and the environment, which is aligned with the principles of a "One Health" approach.

[1]  E. Lingohr,et al.  An Assessment of Antimicrobial Resistant Disease Threats in Canada , 2015, PloS one.

[2]  J. R. Johnson,et al.  Predicting antimicrobial susceptibilities for Escherichia coli and Klebsiella pneumoniae isolates using whole genomic sequence data , 2013, The Journal of antimicrobial chemotherapy.

[3]  G. Dantas,et al.  Next-generation approaches to understand and combat the antibiotic resistome , 2017, Nature Reviews Microbiology.

[4]  Jun Lin,et al.  Mechanisms of antibiotic resistance , 2015, Front. Microbiol..

[5]  A. MacGowan,et al.  Widespread implementation of EUCAST breakpoints for antibacterial susceptibility testing in Europe. , 2015, Euro surveillance : bulletin Europeen sur les maladies transmissibles = European communicable disease bulletin.

[6]  Jianzhong Shen,et al.  Emergence of plasmid-mediated colistin resistance mechanism MCR-1 in animals and human beings in China: a microbiological and molecular biological study. , 2015, The Lancet. Infectious diseases.

[7]  Justin Chu,et al.  NanoSim: nanopore sequence read simulator based on statistical characterization , 2016, bioRxiv.

[8]  Daniel J. Wilson,et al.  Prediction of Staphylococcus aureus Antimicrobial Resistance by Whole-Genome Sequencing , 2014, Journal of Clinical Microbiology.

[9]  Van Den Eede Guy,et al.  The Role and Implementation of Next-Generation Sequencing Technologies in the Coordinated Action Plan against Antimicrobial Resistance , 2017 .

[10]  Kok-Gan Chan,et al.  Complete genome sequencing revealed novel genetic contexts of the mcr-1 gene in Escherichia coli strains , 2016, The Journal of antimicrobial chemotherapy.

[11]  Lisa C. Crossman,et al.  Identification of bacterial pathogens and antimicrobial resistance directly from clinical urines by nanopore-based metagenomic sequencing , 2016, The Journal of antimicrobial chemotherapy.

[12]  Daniel J. Wilson,et al.  A pilot study of rapid benchtop sequencing of Staphylococcus aureus and Clostridium difficile for outbreak detection and surveillance , 2012, BMJ Open.

[13]  Weida Tong,et al.  Baseline Practices for the Application of Genomic Data Supporting Regulatory Food Safety. , 2017, Journal of AOAC International.

[14]  Johan Bengtsson-Palme,et al.  Antibiotic resistance genes in the environment: prioritizing risks , 2015, Nature Reviews Microbiology.

[15]  Bing Li,et al.  Fate of antibiotic resistance genes in sewage treatment plant revealed by metagenomic approach. , 2014, Water research.

[16]  S. Forster Illuminating microbial diversity , 2017, Nature Reviews Microbiology.

[17]  John D Pfeifer,et al.  In Silico Proficiency Testing for Clinical Next-Generation Sequencing. , 2017, The Journal of molecular diagnostics : JMD.

[18]  G. Kahlmeter Defining antibiotic resistance-towards international harmonization , 2014, Upsala journal of medical sciences.

[19]  B. Lilje,et al.  High Interlaboratory Reproducibility and Accuracy of Next-Generation-Sequencing-Based Bacterial Genotyping in a Ring Trial , 2017, Journal of Clinical Microbiology.

[20]  H. Swerdlow,et al.  A tale of three next generation sequencing platforms: comparison of Ion Torrent, Pacific Biosciences and Illumina MiSeq sequencers , 2012, BMC Genomics.

[21]  Christoph Endrullat Standardization in next-generation sequencing - Issues and approaches of establishing standards in a highly dynamic environment , 2017 .

[22]  Ümit V. Çatalyürek,et al.  Benchmarking short sequence mapping tools , 2013, BMC Bioinformatics.

[23]  Christoph Endrullat,et al.  Standardization and quality management in next-generation sequencing , 2016, Applied & translational genomics.

[24]  Peter M. Rice,et al.  The Sanger FASTQ file format for sequences with quality scores, and the Solexa/Illumina FASTQ variants , 2009, Nucleic acids research.

[25]  Robert Schlaberg,et al.  Validation of Metagenomic Next-Generation Sequencing Tests for Universal Pathogen Detection. , 2017, Archives of pathology & laboratory medicine.

[26]  Shashikant Kulkarni,et al.  Good laboratory practice for clinical next-generation sequencing informatics pipelines , 2015, Nature Biotechnology.

[27]  E. Kristiansson,et al.  Environmental factors influencing the development and spread of antibiotic resistance , 2017, FEMS microbiology reviews.

[28]  N Woodford,et al.  The role of whole genome sequencing in antimicrobial susceptibility testing of bacteria: report from the EUCAST Subcommittee. , 2017, Clinical microbiology and infection : the official publication of the European Society of Clinical Microbiology and Infectious Diseases.

[29]  J. Slonczewski,et al.  A Requirement of TolC and MDR Efflux Pumps for Acid Adaptation and GadAB Induction in Escherichia coli , 2011, PloS one.

[30]  F. Baquero,et al.  Tackling antibiotic resistance: the environmental framework , 2015, Nature Reviews Microbiology.

[31]  Teresa M. Coque,et al.  What is a resistance gene? Ranking risk in resistomes , 2014, Nature Reviews Microbiology.

[32]  Robert J. Clifford,et al.  The Challenges of Implementing Next Generation Sequencing Across a Large Healthcare System, and the Molecular Epidemiology and Antibiotic Susceptibilities of Carbapenemase-Producing Bacteria in the Healthcare System of the U.S. Department of Defense , 2016, PloS one.

[33]  Kai Zhou,et al.  Application of next generation sequencing in clinical microbiology and infection prevention. , 2017, Journal of biotechnology.

[34]  Molly K. Gibson,et al.  Improved annotation of antibiotic resistance determinants reveals microbial resistomes cluster by ecology , 2014, The ISME Journal.

[35]  Jinyang Zhao,et al.  Genome sequencing of the sweetpotato whitefly Bemisia tabaci MED/Q , 2017, GigaScience.

[36]  C. Carrillo,et al.  Genomic Tools for Customized Recovery and Detection of Foodborne Shiga Toxigenic Escherichia coli. , 2016, Journal of food protection.

[37]  C. Furusawa,et al.  Comparison of Sequence Reads Obtained from Three Next-Generation Sequencing Platforms , 2011, PloS one.

[38]  Phelim Bradley,et al.  Rapid antibiotic-resistance predictions from genome sequence data for Staphylococcus aureus and Mycobacterium tuberculosis , 2015, Nature Communications.

[39]  A P MacGowan,et al.  EUCAST expert rules in antimicrobial susceptibility testing. , 2013, Clinical microbiology and infection : the official publication of the European Society of Clinical Microbiology and Infectious Diseases.

[40]  Nafees Qamar,et al.  Anonymously Analyzing Clinical Datasets , 2015, ArXiv.

[41]  Philip D. Blood,et al.  Critical Assessment of Metagenome Interpretation—a benchmark of metagenomics software , 2017, Nature Methods.

[42]  Jukka Corander,et al.  Whole-Genome Sequencing for Routine Pathogen Surveillance in Public Health: a Population Snapshot of Invasive Staphylococcus aureus in Europe , 2016, mBio.

[43]  Justin Zobel,et al.  SRST2: Rapid genomic surveillance for public health and hospital microbiology labs , 2014, bioRxiv.

[44]  P. Savelkoul,et al.  Presence of mcr-1-positive Enterobacteriaceae in retail chicken meat but not in humans in the Netherlands since 2009. , 2016, Euro surveillance : bulletin Europeen sur les maladies transmissibles = European communicable disease bulletin.

[45]  J. Ronholm,et al.  Metagenomics: The Next Culture-Independent Game Changer , 2017, Front. Microbiol..

[46]  F. Baquero,et al.  Genomic and metagenomic technologies to explore the antibiotic resistance mobilome , 2017, Annals of the New York Academy of Sciences.

[47]  D. Maskell,et al.  Search Engine for Antimicrobial Resistance: A Cloud Compatible Pipeline and Web Interface for Rapidly Detecting Antimicrobial Resistance Genes Directly from Sequence Data , 2015, PloS one.

[48]  Ole Lund,et al.  Benchmarking of methods for identification of antimicrobial resistance genes in bacterial whole genome data. , 2016, The Journal of antimicrobial chemotherapy.

[49]  Leping Li,et al.  ART: a next-generation sequencing read simulator , 2012, Bioinform..

[50]  Michael C. Schatz,et al.  Teaser: Individualized benchmarking and optimization of read mapping results for NGS data , 2015, bioRxiv.

[51]  N. Khardori,et al.  Evaluation of regional antibiograms to monitor antimicrobial resistance in hampton roads, Virginia , 2015, Annals of Clinical Microbiology and Antimicrobials.

[52]  S. Tringe,et al.  High-Throughput Metagenomic Technologies for Complex Microbial Community Analysis: Open and Closed Formats , 2015, mBio.

[53]  M. Struelens,et al.  Survey on the Use of Whole-Genome Sequencing for Infectious Diseases Surveillance: Rapid Expansion of European National Capacities, 2015–2016 , 2017, Front. Public Health.

[54]  R. Kaas,et al.  Detection of mcr-1 encoding plasmid-mediated colistin-resistant Escherichia coli isolates from human bloodstream infection and imported chicken meat, Denmark 2015. , 2015, Euro surveillance : bulletin Europeen sur les maladies transmissibles = European communicable disease bulletin.

[55]  Gonçalo R. Abecasis,et al.  The Sequence Alignment/Map format and SAMtools , 2009, Bioinform..

[56]  Dahui Qin,et al.  Multi-Institutional FASTQ File Exchange as a Means of Proficiency Testing for Next-Generation Sequencing Bioinformatics and Variant Interpretation. , 2016, The Journal of molecular diagnostics : JMD.

[57]  Dominique Lavenier,et al.  Quality Metrics for Benchmarking Sequences Comparison Tools , 2014, BSB.

[58]  J Moran-Gilad,et al.  Practical issues in implementing whole-genome-sequencing in routine diagnostic microbiology. , 2017, Clinical microbiology and infection : the official publication of the European Society of Clinical Microbiology and Infectious Diseases.

[59]  R. Evans European Centre for Disease Prevention and Control. , 2014, Nursing standard (Royal College of Nursing (Great Britain) : 1987).

[60]  Shashikant Kulkarni,et al.  Assuring the quality of next-generation sequencing in clinical laboratory practice , 2012, Nature Biotechnology.

[61]  Anna Shcherbina,et al.  FASTQSim: platform-independent data characterization and in silico read generation for NGS datasets , 2014, BMC Research Notes.

[62]  Mohamad Adam Bujang,et al.  Requirements for Minimum Sample Size for Sensitivity and Specificity Analysis. , 2016, Journal of clinical and diagnostic research : JCDR.

[63]  J Moran-Gilad,et al.  Microbial Metagenomics Mock Scenario-based Sample Simulation (M3S3). , 2018, Clinical microbiology and infection : the official publication of the European Society of Clinical Microbiology and Infectious Diseases.

[64]  S. Kennedy,et al.  In-depth resistome analysis by targeted metagenomics , 2017, Microbiome.

[65]  M. Webber,et al.  Molecular mechanisms of antibiotic resistance , 2014, Nature Reviews Microbiology.

[66]  Herman Goossens,et al.  Consolidating and Exploring Antibiotic Resistance Gene Data Resources , 2016, Journal of Clinical Microbiology.

[67]  N. Makridakis,et al.  Whole-genome sequencing targets drug-resistant bacterial infections , 2015, Human Genomics.

[68]  Kiyoshi Asai,et al.  PBSIM: PacBio reads simulator - toward accurate genome assembly , 2013, Bioinform..

[69]  Hua Yang,et al.  Characterization of the resistome in manure, soil and wastewater from dairy and beef production systems , 2016, Scientific Reports.

[70]  E. Kristiansson,et al.  Using metagenomics to investigate human and environmental resistomes , 2017, The Journal of antimicrobial chemotherapy.

[71]  Michael C. Schatz,et al.  Third-generation sequencing and the future of genomics , 2016, bioRxiv.

[72]  J. O’Grady,et al.  The variability and reproducibility of whole genome sequencing technology for detecting resistance to anti-tuberculous drugs , 2016, Genome Medicine.

[73]  Paul P. Gardner,et al.  An evaluation of the accuracy and speed of metagenome analysis tools , 2015, Scientific Reports.

[74]  J. Rolain,et al.  ARG-ANNOT, a New Bioinformatic Tool To Discover Antibiotic Resistance Genes in Bacterial Genomes , 2013, Antimicrobial Agents and Chemotherapy.

[75]  T. Seemann,et al.  Outbreak Investigation Using High-Throughput Genome Sequencing within a Diagnostic Microbiology Laboratory , 2013, Journal of Clinical Microbiology.

[76]  P. Toutain,et al.  En Route towards European Clinical Breakpoints for Veterinary Antimicrobial Susceptibility Testing: A Position Paper Explaining the VetCAST Approach , 2017, Front. Microbiol..

[77]  Damiano Cacace,et al.  Seasonality of antibiotic prescriptions for outpatients and resistance genes in sewers and wastewater treatment plant outflow. , 2016, FEMS microbiology ecology.

[78]  Erik Kristiansson,et al.  Shotgun metagenomics reveals a wide array of antibiotic resistance genes and mobile elements in a polluted lake in India , 2014, Front. Microbiol..