Forensic audio and visual evidence 2004-2007 : A Review

Although audio and visual evidence (video, photographs and laserscans) may have been treated by the same experts in many organizations in the past, it is now clear that there are a number of totally different fields of expertise that deal with these types of evidence. In this review, we distinguish six fields of expertise: Audio analysis, Speaker identification, Forensic linguistics, Video analysis, Photogrammetry and 3D modeling, and Facial identification. However, experts on investigations of authenticity and integrity of audio and video still share a number of methods and can benefit from each others knowledge and expertise. Since most of this evidence is now generated as digital data, more expertise on digital evidence is needed. Experts on facial identification, speaker identification and forensic linguistics share a common interest in the use of statistics, and methods for dealing with subjective information. Audio analysis: The new methods for investigation of authentication and integrity, based on the electric network frequency and the use of opto-magnetic crystals, that have been introduced in the period of the review 2001-2004, are well known now, and more reports are being published on the use and effectiveness of these methods. Speaker identification: In the period of this review a lot of research has been done on the use of statistics of frequency measurements in acoustic analysis. Forensic speaker identification is now preferably based on a combination of results from auditory analysis and acoustic analysis as well, using a Bayesian framework for assessing the evidential value. Forensic linguistics: this expertise is now often requested for in the analysis of letters claiming responsibility for politically motivated offenses, and the language samples from refugees in order to confirm their alleged origin. The use of text databases for statistical analysis is being debated. Video analysis: the widespread introduction of large scale digital video surveillance systems in public and private domains resulted in large scale research and development programs and the development of a new field of expertise closely linked to the development of special organizations for dealing with digital evidence. Photogrammetry and 3D modeling: laser scanning of crime scenes has become well known technology. The widespread use is still hampered by the high costs of equipment and training of personnel. A large number of papers have been published now on body length estimation from CCTV images. Measurement errors have been studied and quantified. The evidential value is strongly limited by the number of uncertainties in a case. Facial identification: error rates between 5 and 10 % for facial image recognition and identification have been reported for biometric systems and human observers. The latest studies show that human observers who are assisted by biometric systems perform significantly better. For facial reconstruction from the remains of skulls, computer modeling software is available that can work with statistical data on soft tissue thickness measurements. In this review paper a full overview is given on all relevant developments in these fields of discipline, based on an extensive search in literature databases and the exchange of information in a large number of conferences. Review on Forensic audio and Visual Evidence 2004-2007 15th INTERPOL Forensic Science Symposium, Lyon, France, October 2007 Introduction In the review on audio and visual evidence 2001-2004 Two separate papers were presented. The first paper presented a very detailed description of the fields of expertise in (1) audio analysis, (2) speaker identification and (3) linguistic authorship. The second paper presented a general overview of the many different types of forensic investigations on visual evidence and a way to catagorize them into three general fields of expertise: (4) imaging and video technology (5) crime scene photography, laserscanning, photogrammetry and 3d-modeling, and (6) biometric identification. Two other discplines, medical imaging and pattern recognition in forensic database were treated as miscellenious topics in a separate chapter. In this review, all topics in audio and visual evidence are covered in one paper, and two of the previously mentioned fields of expertise will be named differently. The most important developments will be presented for six general fields of expertise: (1)audio analysis, (2) speaker identification, (3) linguistic authorship, (4) imaging and video technology, (5) photogrammetry, crime scene recording and 3dmodeling, (6) facial image identification and earprints. In a separate chapter, we focus briefly on a number of forensic activities that do no fit well in the previous chapters. The last chapter deals with relevant organizations and their work in forensic image and audio analysis. Each chapter starts with a list of keywords to help the reader finding topics of interest. Literature references are given at the end of each chapter. This review is based on information from an extensive search in literature databases (Inspec, Compendex, Scopus, patent databases and several others), participation in meetings organized by the AAFS, ENFSI and SPIE, and contacts with the working groups SWGIT, LEVA, EESAG, ENFSIDIWG and ENFSIAS. This review is certainly not complete for two reasons: most of the information used is obtained from American and European sources and the scope of the review is limited to the fields of expertise that the authors have been working in or with. Due to the amount of publications in certain fields, the authors have not read the complete articles for making this overview. However, for a large number of articles, abstracts provided by the literature database could be used. Audio analysis (Michael Jessen) authentication, speech enhancement, transcription of linguistic content / disputed utterance examination, non-speech events, magneto-optical methods, ENF, Electric Network Frequency The primary domains of forensic audio analysis are authentication, speech enhancement, transcription of linguistic content / disputed utterance examination, and the analysis of non-speech events. For the authentication of analog recordings on audio tape, the use of magneto-optical investigations based on the Faraday or Kerr effect has been recognized by now as the most efficient, accurate and non-destructive method (Boss et al. 2003; Bouten et al. 2007). This method makes use of crystals that contain ferrimagnetic garnet film. When such a crystal is brought into contact with the audio tape it captures the magnetization patterns on the tape very accurately. Viewed under polarized light, these patterns can be examined and photographed. Various types of crystal are available as well as different setups of how the tape is placed and how the crystals are examined. A recent development is MOSeS (magneto-optical sensor system). This system is more practical than previous ones by offering better tape transport (hence easier processing of larger amounts of tape) and more flexible processing of a wider variety of tapes beyond regular compact audio cassettes, including microcassettes and video tapes (Boss 2005). Magneto-optical-based authentication can detect various types of information, including erase head marks, which can indicate if a passage has been deleted, and unusual track widths and positions due to misaligned audio heads etc., which can help to individualize the recorder that was used. Although the general principles of the magneto-optical method are known, many of the specifics of the observable patterns have been interpreted on the basis of experience rather than on a deep theoretical understanding of the physical explanations behind these patterns. An important improvement upon this state of the art has been provided recently by Bouten et al. (2007), who presents a detailed theory of magneto-optical authentications and tested its validity with experiments where the Review on Forensic audio and Visual Evidence 2004-2007 15th INTERPOL Forensic Science Symposium, Lyon, France, October 2007 audio data and recording devices are known. At the 2005 meeting of the ENFSI (European Network of Forensic Science Institutes) Working Group for Forensic Speech and Audio Analysis (FSAAWG) in Wiesbaden the results of a collaborative exercise on analog audio authentication were presented, showing, among other things, that the use of magneto-optical methods is time-consuming but that is offers advantages not achievable by other methods. Whereas methods for the authentication of analog recordings on tape are well established, still extremely little has been published on authentication of digital audio (see the overview of Cooper 2006). Although tampering might leave no traces if performed by somebody with profound knowledge in areas such as signal analysis, acoustics and phonetics, and if the original material is of sufficient quality, tampering is more than a simple cut and paste operation (Cooper 2005). For example, since the recordings are rarely in studio quality they will probably contain background noise, and cutting or copying sections of speech might lead to noticeable discontinuities. With authentication of digital speech data there is a stronger burden on linguistic and phonetic methods than before. The general idea of these methods is that additions or deletions might have taken place where the sequence of linguistic and phonetic events in a recording does not correspond to the known rules of linguistic and phonetic sequencing. For example, the sequences of words might not correspond to syntactic rules, or the sequences of sounds and sound elements might not be compatible with the rules of phonotactics or coarticulation. Application of this method requires good knowledge of the features of spontaneous fluent speech, since the phonological and phonetic sequencing rules in this speech style can differ, for examp