Intuition versus Algorithm: The Case of Forensic Authorship Attribution

This article addresses a nagging issue in the field of scientific evidence: What should the legal system do when experts developing a statistical approach to forensic identification are making good progress, but are not provably more accurate than experts who make judgments, often convincing judgments, based upon the their analysis of the specific facts of each case? That is the state of affairs in the field of authorship attribution: Computer scientists and computational linguists develop and test their models while a group of forensic linguists continues to testify in cases without mathematical checks on their conclusions. The legal system rightly prefers algorithmic expertise over intuitive expertise, but when it is not clear that the algorithms do a better job, the question becomes more difficult. The article discusses the psychological literature on the question of algorithm versus intuition and applies it to authorship attribution. It concludes that the insights of the intuitive experts, sometimes called practitioners of forensic stylistics, may have a great deal to contribute to the models created by the computational experts; that practitioners of stylistic comparison have an obligation to conduct far more research into the accuracy of their methods, including, in the short-run, proficiency testing; and that a healthy combination of cooperation and competition is gradually leading to improvements in the field and convergence around those methods that prove successful.