A Review of the Scientific Rigor, Reproducibility, and Transparency Studies Conducted by the ABRF Research Groups.

Shared research resource facilities, also known as core laboratories (Cores), are responsible for generating a significant and growing portion of the research data in academic biomedical research institutions. Cores represent a central repository for institutional knowledge management, with deep expertise in the strengths and limitations of technology and its applications. They inherently support transparency and scientific reproducibility by protecting against cognitive bias in research design and data analysis, and thedy have institutional responsibility for the conduct of research (research ethics, regulatory compliance, and financial accountability) performed in their Cores. The Association of Biomolecular Resource Facilities (ABRF) is a FASEB-member scientific society whose members are scientists and administrators that manage or support Cores. The ABRF Research Groups (RGs), representing expertise for an array of cutting-edge and established technology platforms, perform multicenter research studies to determine and communicate best practices and community-based standards. This review provides a summary of the contributions of the ABRF RGs to promote scientific rigor and reproducibility in Cores from the published literature, ABRF meetings, and ABRF RGs communications.

[1]  Erik Schultes,et al.  The FAIR Guiding Principles for scientific data management and stewardship , 2016, Scientific Data.

[2]  Vincent L. Butty,et al.  Cross-site comparison of ribosomal depletion kits for Illumina RNAseq library construction , 2017, BMC Genomics.

[3]  Shawn Levy,et al.  International Standards for Genomes, Transcriptomes, and Metagenomes. , 2017, Journal of biomolecular techniques : JBT.

[4]  Thomas A Neubert,et al.  MIRG Survey 2011: snapshot of rapidly evolving label-free technologies used for characterizing molecular interactions. , 2012, Journal of biomolecular techniques : JBT.

[5]  A. Yamniuk,et al.  ABRF-MIRG benchmark study: molecular interactions in a three-component system. , 2012, Journal of biomolecular techniques : JBT.

[6]  Julie Gould Core facilities: Shared support. , 2015, Nature.

[7]  R. Angeletti,et al.  Six-year study of peptide synthesis. , 1997, Methods in enzymology.

[8]  G A Buck,et al.  Multi-facility survey of oligonucleotide synthesis and an examination of the performance of unpurified primers in automated DNA sequencing. , 1996, BioTechniques.

[9]  David L. Tabb,et al.  The 2012/2013 ABRF Proteomic Research Group Study: Assessing Longitudinal Intralaboratory Variability in Routine Peptide Liquid Chromatography Tandem Mass Spectrometry Analyses* , 2015, Molecular & Cellular Proteomics.

[10]  Wei Shi,et al.  Detecting and correcting systematic variation in large-scale RNA sequencing data , 2014, Nature Biotechnology.

[11]  K. Medzihradszky,et al.  Strategies for the synthesis of labeled peptides. , 2000, Journal of Biomolecular Techniques.

[12]  Thomas A Neubert,et al.  ABRF-PRG04: differentiation of protein isoforms. , 2007, Journal of biomolecular techniques : JBT.

[13]  Mercè Crosas,et al.  Data Authorship as an Incentive to Data Sharing. , 2017, The New England journal of medicine.

[14]  P. Andrews,et al.  Assignment of cysteine and tryptophan residues during protein sequencing: Results of ABRF-94SEQ , 1995 .

[15]  S. Cohen,et al.  Synthesis of a fluorescent derivatizing reagent, 6-aminoquinolyl-N-hydroxysuccinimidyl carbamate, and its application for the analysis of hydrolysate amino acids via high-performance liquid chromatography. , 1993, Analytical biochemistry.

[16]  Andrew A. Ott,et al.  Building a Sustainable Portfolio of Core Facilities: a Case Study. , 2018, Journal of biomolecular techniques : JBT.

[17]  Y. Mechref,et al.  Glycoproteomics: Identifying the Glycosylation of Prostate Specific Antigen at Normal and High Isoelectric Points by LC–MS/MS , 2014, Journal of proteome research.

[18]  R. Cook,et al.  EDMAN SEQUENCING RESEARCH GROUP 2004 STUDY: MODIFIED AMINO ACIDS IN EDMAN SEQUENCING , 2005 .

[19]  D. J. Strydom,et al.  Sensitive Analysis of Cystine/Cysteine using 6-Aminoquinolyl-N-Hydroxysuccinimidyl Carbamate (AQC) Derivatives , 1993 .

[20]  R. Pepperkok,et al.  Institutional core facilities: prerequisite for breakthroughs in the life sciences , 2016, EMBO reports.

[21]  A. Sporbert,et al.  Advanced light microscopy core facilities: Balancing service, science and career , 2016, Microscopy research and technique.

[22]  Bruce A. Stanley,et al.  Mass Spectrometry Data Collection in Parallel at Multiple Core Facilities Operating TripleTOF 5600 and Orbitrap Elite/Velos Pro/Q Exactive Mass Spectrometers. , 2013 .

[23]  W. T. Moore,et al.  Formation of a Disulfide Bond in an Octreotide-Like Peptide: A Multicenter Study , 1996 .

[24]  M. Lauer,et al.  Reproducibility 2020 : Progress and priorities , 2017 .

[25]  Lennart Martens,et al.  Predicting tryptic cleavage from proteomics data using decision tree ensembles. , 2013, Journal of proteome research.

[26]  ABRF-PRG05: de novo peptide sequence determination. , 2008, Journal of biomolecular techniques : JBT.

[27]  G A Buck,et al.  Accuracy of automated DNA sequencing: a multi-laboratory comparison of sequencing results. , 1995, BioTechniques.

[28]  K. Williams,et al.  10 – A SYNTHETIC PEPTIDE FOR EVALUATING PROTEIN SEQUENCER AND AMINO ACID ANALYZER PERFORMANCE IN CORE FACILITIES: DESIGN AND RESULTS , 1989 .

[29]  Alexa B. R. McIntyre,et al.  Extensive sequencing of seven human genomes to characterize benchmark reference materials , 2015, Scientific Data.

[30]  D G Myszka,et al.  The ABRF-MIRG'02 study: assembly state, thermodynamic, and kinetic analysis of an enzyme/inhibitor interaction. , 2003, Journal of biomolecular techniques : JBT.

[31]  David L Tabb,et al.  Quality assessment for clinical proteomics. , 2013, Clinical biochemistry.

[32]  Gregory K. Farber,et al.  Core Facilities: Maximizing the Return on Investment , 2011, Science Translational Medicine.

[33]  William Arbuthnot Sir Lane,et al.  ABRF-97 SEQ: Sequencing results of a low-level sample , 1999 .

[34]  I. Apostol,et al.  The hydrolysis process and the quality of amino acid analysis: ABRF-94AAA collaborative trial , 1995 .

[35]  I. Apostol,et al.  Cysteine and Tryptophan Amino Acid Analysis of ABRF92-AAA , 1993 .

[36]  G. Shipley,et al.  Evaluation of dual-labeled fluorescent DNA probe purity versus performance in real-time PCR. , 2004, BioTechniques.

[37]  S. Cohen,et al.  Using quaternary high-performance liquid chromatography eluent systems for separating 6-aminoquinolyl-N-hydroxysuccinimidyl carbamate-derivatized amino acid mixtures , 1997 .

[38]  Christoph Endrullat,et al.  Standardization and quality management in next-generation sequencing , 2016, Applied & translational genomics.

[39]  Lennart Martens,et al.  iPRG 2011: A Study on the Identification of Electron Transfer Dissociation (ETD) Mass Spectra , 2011 .

[40]  Christopher E Mason,et al.  Assessment of REPLI-g Multiple Displacement Whole Genome Amplification (WGA) Techniques for Metagenomic Applications. , 2017, Journal of biomolecular techniques : JBT.

[41]  Michael J MacCoss,et al.  ABRF-PRG07: advanced quantitative proteomics study. , 2011, Journal of biomolecular techniques : JBT.

[42]  J. Rush,et al.  Design and analysis of ABRF-95SEQ, a recombinant protein with sequence heterogeneity , 1996 .

[43]  Christopher E. Mason,et al.  The ABRF Next Generation Sequencing Study: Multi-platform and Cross-methodological Reproducibility of Transcriptome Profiling by RNA-seq , 2014 .

[44]  K. Knudtson,et al.  DNA SEQUENCING RESEARCH GROUP (DSRG) 2003—A GENERAL SURVEY OF CORE DNA SEQUENCING FACILITIES. , 2003 .

[45]  D. J. Strydom,et al.  Amino acid analysis utilizing phenylisothiocyanate derivatives. , 1988, Analytical biochemistry.

[46]  R. DeSalle,et al.  Large-scale differences in microbial biodiversity discovery between 16S amplicon and shotgun sequencing , 2017, Scientific Reports.

[47]  C. Mason,et al.  The impact of read length on quantification of differentially expressed genes and splice junction detection , 2015, Genome Biology.

[48]  Samuel H Payne,et al.  A protein standard that emulates homology for the characterization of protein inference algorithms , 2017, bioRxiv.

[49]  Heike Sichtig,et al.  Single-molecule sequencing detection of N6-methyladenine in microbial reference materials , 2019, Nature Communications.

[50]  William Arbuthnot Sir Lane,et al.  Evaluation of ABRF-96SEQ: A sequence assignment exercise , 1997 .

[51]  Henry H N Lam,et al.  iPRG 2012 Study: Detecting Modified Peptides in a Complex Mixture , 2012 .

[52]  M. Zianni,et al.  Identification of optimal protocols for sequencing difficult templates: results of the 2008 ABRF DNA Sequencing Research Group difficult template study 2008. , 2009, Journal of biomolecular techniques : JBT.

[53]  Simon Bergqvist,et al.  Conclusions from the MIRG 2010 Benchmark Study: Molecular Interactions in a Three Component System and Presentation of 2011 Survey Results on Label-Free Technologies. , 2011 .

[54]  Robert L. Moritz,et al.  PRG-2011: Defining the Interaction Between Users and Suppliers of Proteomics Services/Facilities , 2011 .

[55]  Hendrik Weisser,et al.  Targeted Feature Detection for Data-Dependent Shotgun Proteomics , 2017, Journal of proteome research.

[56]  Thomas A Neubert,et al.  The ABRF Metabolomics Research Group 2013 Study: Investigation of Spiked Compound Differences in a Human Plasma Matrix. , 2015, Journal of biomolecular techniques : JBT.

[57]  Reality check on reproducibility , 2016, Nature.

[58]  Noah Alexander,et al.  Precision Metagenomics: Rapid Metagenomic Analyses for Infectious Disease Diagnostics and Public Health Surveillance. , 2017, Journal of biomolecular techniques : JBT.

[59]  Alexa B. R. McIntyre,et al.  Genomic Methods and Microbiological Technologies for Profiling Novel and Extreme Environments for the Extreme Microbiome Project (XMP). , 2017, Journal of biomolecular techniques : JBT.

[60]  Lynda F. Bonewald,et al.  Correlation of cleavage techniques with side-reactions following solid-phase peptide synthesis , 1995 .

[61]  P. Hunziker,et al.  Identification of proteins electroblotted to polyvinylidene difluoride membrane by combined amino acid analysis and bioinformatics: An ABRF multicenter study. , 1999, Journal of biomolecular techniques : JBT.

[62]  C. Turck,et al.  The Association of Biomolecular Resource Facilities Proteomics Research Group 2006 Study , 2007, Molecular & Cellular Proteomics.

[63]  Jeroen Krijgsveld,et al.  The ABRF Proteomics Research Group Studies: Educational exercises for qualitative and quantitative proteomic analyses , 2011, Proteomics.

[64]  L. C. Williams,et al.  Variable Success of Peptide-Resin Cleavage and Deprotection Following Solid-Phase Synthesis , 1994 .

[65]  K. Knudtson,et al.  Evaluation of methods for sequence analysis of highly repetitive DNA templates. , 2006, Journal of biomolecular techniques : JBT.

[66]  Henry H N Lam,et al.  Proteome Informatics Research Group (iPRG)_2012: A Study on Detecting Modified Peptides in a Complex Mixture* , 2013, Molecular & Cellular Proteomics.

[67]  James M. Anderson,et al.  U.S. National Institutes of Health core consolidation-investing in greater efficiency. , 2015, Journal of biomolecular techniques : JBT.

[68]  Peter Lopez,et al.  The New ABRF Flow Cytometry Research Group (FCRG). , 2013 .

[69]  A. Yamniuk,et al.  Development of a Model Protein Interaction Pair as a Benchmarking Tool for the Quantitative Analysis of 2-Site Protein-Protein Interactions. , 2015, Journal of biomolecular techniques : JBT.

[70]  Anne-Marie Girard,et al.  Quality assurance testing for modern optical imaging systems. , 2011, Microscopy and microanalysis : the official journal of Microscopy Society of America, Microbeam Analysis Society, Microscopical Society of Canada.

[71]  W. Sandoval,et al.  The ABRF Edman Sequencing Research Group 2008 Study: investigation into homopolymeric amino acid N-terminal sequence tags and their effects on automated Edman degradation. , 2009, Journal of biomolecular techniques : JBT.

[72]  Use of mass spectrometry, capillary electrophoresis, and gel electrophoresis for quality analysis of synthetic oligonucleotides: Perspectives from the ABRF nucleic acid research group. , 2001 .

[73]  Lloyd M. Smith,et al.  How many human proteoforms are there? , 2018, Nature chemical biology.

[74]  Douglas J. Botkin,et al.  Nanopore DNA Sequencing and Genome Assembly on the International Space Station , 2016, bioRxiv.

[75]  R. Paxton,et al.  Collaborative Trial Analyses of ABRF-91AAA , 1992 .

[76]  Samuel H Payne,et al.  ABRF Proteome Informatics Research Group (iPRG) 2016 Study: Inferring Proteoforms from Bottom-up Proteomics Data. , 2018, Journal of biomolecular techniques : JBT.

[77]  Thomas A Neubert,et al.  ABRF-PRG03: phosphorylation site determination. , 2003, Journal of biomolecular techniques : JBT.

[78]  Ekaterina V. Poverennaya,et al.  The Size of the Human Proteome: The Width and Depth , 2016, International journal of analytical chemistry.

[79]  Robert G. Steen,et al.  Comparison of commercially available target enrichment methods for next-generation sequencing. , 2013, Journal of biomolecular techniques : JBT.

[80]  Sheng Li,et al.  Multi-platform assessment of transcriptome profiling using RNA-seq in the ABRF next-generation sequencing study , 2014, Nature Biotechnology.

[81]  Brendan MacLean,et al.  ABRF Proteome Informatics Research Group (iPRG) 2015 Study: Detection of Differentially Abundant Proteins in Label-Free Quantitative LC-MS/MS Experiments. , 2017, Journal of proteome research.

[82]  P S Adams,et al.  Design strategies and performance of custom DNA sequencing primers. , 1999, BioTechniques.

[83]  Claire M Brown,et al.  International Test Results for Objective Lens Quality, Resolution, Spectral Accuracy and Spectral Separation for Confocal Laser Scanning Microscopes , 2013, Microscopy and Microanalysis.

[84]  Brendan MacLean,et al.  ABRF Research Group Development and Characterization of a Proteomics Normalization Standard Consisting of 1,000 Stable Isotope Labeled Peptides , 2014 .

[85]  G A Buck,et al.  A survey of nucleic acid services in core laboratories. , 1994, BioTechniques.

[86]  W. T. Moore,et al.  Analysis of racemization during "Standard" solid phase peptide synthesis: a multicenter study , 1997 .

[87]  N M Ritter,et al.  Analytical laboratory quality: Part I. General quality practices. , 2001, Journal of biomolecular techniques : JBT.

[88]  Emmanuel Dias-Neto,et al.  The Metagenomics and Metadesign of the Subways and Urban Biomes (MetaSUB) International Consortium inaugural meeting report , 2016, Microbiome.

[89]  I. Cockburn,et al.  The Economics of Reproducibility in Preclinical Research , 2015, PLoS biology.

[90]  Howard Gadlin,et al.  Collaboration and Team Science , 2012, Journal of Investigative Medicine.

[91]  Brian A. Nosek,et al.  Promoting an open research culture , 2015, Science.

[92]  J. Ioannidis,et al.  Reproducibility in Science: Improving the Standard for Basic and Preclinical Research , 2015, Circulation research.

[93]  R. Thoma,et al.  ABRF ESRG 2006 study: Edman sequencing as a method for polypeptide quantitation. , 2007, Journal of biomolecular techniques : JBT.

[94]  P. Urban,et al.  Quantitative mass spectrometry: an overview , 2016, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[95]  S. Carr,et al.  ABRF-98SEQ: Evaluation of peptide sequencing at high sensitivity. , 2000, Journal of biomolecular techniques : JBT.

[96]  F. Collins,et al.  Policy: NIH plans to enhance reproducibility , 2014, Nature.

[97]  A. Viale,et al.  Big results from small samples: evaluation of amplification protocols for gene expression profiling. , 2007, Journal of biomolecular techniques : JBT.

[98]  Journals unite for reproducibility , 2014, Nature.

[99]  P. Andrews,et al.  PROTEIN SEQUENCING OF POST-TRANSLATIONALLY MODIFIED PEPTIDES and PROTEINS: DESIGN, CHARACTERIZATION AND RESULTS OF ABRF-92SEQ , 1993 .

[100]  F. Grieder,et al.  Sharing Core Facilities and Research Resources--An Investment in Accelerating Scientific Discoveries. , 2016, Journal of biomolecular techniques : JBT.

[101]  D. J. Strydom,et al.  Comparison of amino acid analyses by phenylisothiocyanate and 6-aminoquinolyl-N-hydroxysuccinimidyl carbamate precolumn derivatization. , 1994, Analytical biochemistry.

[102]  John M. Asara,et al.  Metabolomics Research Group 2011 Study. , 2011 .

[103]  R. Cook,et al.  ABRF-ESRG'03: analysis of a PVDF-bound known protein with a homogeneous amino-terminus. , 2003, Journal of biomolecular techniques : JBT.

[104]  John P. A. Ioannidis,et al.  What does research reproducibility mean? , 2016, Science Translational Medicine.

[105]  Karl Mechtler,et al.  Interlaboratory studies and initiatives developing standards for proteomics , 2013, Proteomics.

[106]  S. Green,et al.  Next Generation Sequencing and the Extreme Microbiome Project (XMP) , 2015 .

[107]  Christopher E. Mason,et al.  The ABRF-Next Generation Sequencing Study: A Five-Platform, Cross-site, Cross-Protocol Examination of RNA Sequencing , 2013 .

[108]  P. Andrews,et al.  A Synthetic Peptide for Evaluating Protein Sequencing Capabilities: Design of ABRF-93SEQ and Results , 1994 .

[109]  K. Dewar,et al.  Sequencing of the Dutch elm disease fungus genome using the Roche/454 GS-FLX Titanium System in a comparison of multiple genomics core facilities. , 2013, Journal of biomolecular techniques : JBT.

[110]  I. Apostol,et al.  Amino Acid Analysis of Phospho-Peptides: Abrf-93AAA , 1994 .

[111]  Frederick J. Manning,et al.  Resource Sharing in Biomedical Research , 1996 .

[112]  S A Cohen,et al.  Applications of amino acid derivatization with 6-aminoquinolyl-N-hydroxysuccinimidyl carbamate. Analysis of feed grains, intravenous solutions and glycoproteins. , 1994, Journal of chromatography. A.

[113]  H Shindo,et al.  Nucleic Acids , 1932, Nature.

[114]  Michael Zianni,et al.  A Methodology Study for Metagenomics Using Next Generation Sequencers , 2011 .

[115]  James Inglese,et al.  The increasing urgency for standards in basic biologic research. , 2014, Cancer research.

[116]  J. McMurray,et al.  Study on the synthesis and characterization of peptides containing phosphorylated tyrosine. , 1999, The journal of peptide research : official journal of the American Peptide Society.

[117]  David W. Speicher,et al.  15 – Design, Characterization and Results of ABRF-89SEQ: A Test Sample For Evaluating Protein Sequencer Performance in Protein Microchemistry Core Facilities , 1990 .

[118]  William Arbuthnot Sir Lane,et al.  ABRF ESRG 2005 study: identification of seven modified amino acids by Edman sequencing. , 2006, Journal of biomolecular techniques : JBT.

[119]  Scott D Buckel,et al.  ABRF-2002ESRG, a difficult sequence: Analysis of a PVDF-bound known protein with a heterogeneous amino-terminus. , 2002, Journal of biomolecular techniques : JBT.

[120]  David P. Kreil,et al.  A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control consortium , 2014, Nature Biotechnology.

[121]  William Arbuthnot Sir Lane,et al.  Proteomics in mixtures: Study results of ABRF-PRG02. , 2002, Journal of biomolecular techniques : JBT.

[122]  Haixu Tang,et al.  Interlaboratory Study on Differential Analysis of Protein Glycosylation by Mass Spectrometry: The ABRF Glycoprotein Research Multi-Institutional Study 2012* , 2013, Molecular & Cellular Proteomics.

[123]  J. McMurray,et al.  1997 ABRF Peptide synthesis research committee study on a peptide containing a phosphorylated tyrosine , 2002 .

[124]  John P. A. Ioannidis,et al.  A manifesto for reproducible science , 2017, Nature Human Behaviour.

[125]  Rebecca L Hegstad-Davies,et al.  Survey on Scientific Shared Resource Rigor and Reproducibility. , 2019, Journal of biomolecular techniques : JBT.

[126]  R. Angeletti,et al.  Research Technologies: Fulfilling the Promise 1 , 1999, FASEB journal : official publication of the Federation of American Societies for Experimental Biology.

[127]  K. Williams,et al.  INTERNAL PROTEIN SEQUENCING OF SDS PAGE-SEPARATED PROTEINS: A COLLABORATIVE ABRF STUDY , 1997 .

[128]  K. Williams,et al.  Amino Acid Analysis and Sequencing — What is State-of-the-Art? , 1991 .

[129]  J. W. Fox,et al.  Amino acid analysis—recovery from PVDF membranes: ABRF-95AAA collaborative trial , 1996 .

[130]  N. Steneck ORI Introduction to Responsible Conduct of Research , 2007 .

[131]  Claire M. Brown,et al.  Measuring and interpreting point spread functions to determine confocal microscope resolution and ensure quality control , 2011, Nature Protocols.

[132]  C. Slaughter,et al.  Evaluation of Protein Sequencing Core Facilities: Design, Characterization, and Results from a Test Sample (ABRF-91SEQ) , 1992 .

[133]  Cassandra Willyard New human gene tally reignites debate , 2018, Nature.