Novel algorithms and the benefits of comparative validation

Contact: 2robsmith@gmail.com, jtprince@chem.byu.edu Bioinformatic research has produced a large volume of proposed algorithmic solutions to a host of problems. Whether presented as a processing step in a clinical experiment or treated in a stand-alone publication, novel bioinformatic algorithms are often not subjected to the thorough comparative evaluation endured by their counterparts in other closely related fields—such as computer science—where an algorithm unevaluated against extant methods is considered unpublishable. Two audiences are interested in algorithmic publications: the practitioner, who may use the algorithm, and the researcher, who will work to develop solutions superior to those extant. We argue that failure during the review/publication process to require comparative evaluation for novel algorithms is detrimental to both parties. To demonstrate the dilemma, we conducted a case study of novel LC-MS alignment algorithms. Of the 48 publications from 2001 to 2012 that present alignment algorithms of which we are aware, 60% include no comparison to other methods. Another 20%