Matching Forensic Sketches to Mug Shot Photos

The problem of matching a forensic sketch to a gallery of mug shot images is addressed in this paper. Previous research in sketch matching only offered solutions to matching highly accurate sketches that were drawn while looking at the subject (viewed sketches). Forensic sketches differ from viewed sketches in that they are drawn by a police sketch artist using the description of the subject provided by an eyewitness. To identify forensic sketches, we present a framework called local feature-based discriminant analysis (LFDA). In LFDA, we individually represent both sketches and photos using SIFT feature descriptors and multiscale local binary patterns (MLBP). Multiple discriminant projections are then used on partitioned vectors of the feature-based representation for minimum distance matching. We apply this method to match a data set of 159 forensic sketches against a mug shot gallery containing 10,159 images. Compared to a leading commercial face recognition system, LFDA offers substantial improvements in matching forensic sketches to the corresponding face images. We were able to further improve the matching performance using race and gender information to reduce the target gallery size. Additional experiments demonstrate that the proposed framework leads to state-of-the-art accuracys when matching viewed sketches.

[1]  Shaun J. Canavan,et al.  A biometric database with rotating head videos and hand-drawn face sketches , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[2]  Hua Yu,et al.  A direct LDA algorithm for high-dimensional data - with application to face recognition , 2001, Pattern Recognit..

[3]  Hanqing Lu,et al.  A nonlinear approach for face sketch synthesis and recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  Wei Liu,et al.  Bayesian Tensor Inference for Sketch-Based Facial Photo Hallucination , 2007, IJCAI.

[5]  B. K. Julsing,et al.  Face Recognition with Local Binary Patterns , 2012 .

[6]  Paul Miller,et al.  Verification of face identities from images captured on video. , 1999 .

[7]  Arun Ross,et al.  Information fusion in biometrics , 2003, Pattern Recognit. Lett..

[8]  Dahua Lin,et al.  Inter-modality Face Recognition , 2006, ECCV.

[9]  Arun Ross,et al.  An introduction to biometric recognition , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Shengcai Liao,et al.  Heterogeneous Face Recognition from Local Structures of Normalized Appearance , 2009, ICB.

[11]  Xiaogang Wang,et al.  Face sketch synthesis and recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[12]  Xuelong Li,et al.  Local face sketch synthesis learning , 2008, Neurocomputing.

[13]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[14]  Norbert Krüger,et al.  Face recognition by elastic bunch graph matching , 1997, Proceedings of International Conference on Image Processing.

[15]  Stan Z. Li,et al.  Coupled Spectral Regression for matching heterogeneous faces , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Anil K. Jain,et al.  Soft Biometric Traits for Personal Recognition Systems , 2004, ICBA.

[17]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Anil K. Jain,et al.  Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[20]  Anil K. Jain,et al.  Sketch-to-photo matching: a feature-based approach , 2010, Defense + Commercial Sensing.

[21]  Xiaogang Wang,et al.  Face sketch recognition , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Amit R.Sharma,et al.  Face Photo-Sketch Synthesis and Recognition , 2012 .

[23]  Vicki Bruce,et al.  The relative importance of external and internal features of facial composites. , 2007, British journal of psychology.

[24]  Niels da Vitoria Lobo,et al.  A framework for recognizing a facial image from a police sketch , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Lior Wolf,et al.  Using Biologically Inspired Features for Face Processing , 2007, International Journal of Computer Vision.

[26]  A. Young,et al.  Matching Familiar and Unfamiliar Faces on Internal and External Features , 1985, Perception.

[27]  Anil K. Jain,et al.  Heterogeneous Face Recognition: Matching NIR to Visible Light Images , 2010, 2010 20th International Conference on Pattern Recognition.

[28]  Xuelong Li,et al.  A new approach for face recognition by sketches in photos , 2009, Signal Process..

[29]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[30]  K. Taylor.  Forensic Art and Illustration , 2000 .

[31]  Chunna Tian,et al.  Face Sketch Synthesis using E-HMM and Selective Ensemble , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[32]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[33]  Xiaogang Wang,et al.  Random sampling LDA for face recognition , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[34]  Y. Hung,et al.  Face verification and identification using Facial Trait Code , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Jiri Matas,et al.  XM2VTSDB: The Extended M2VTS Database , 1999 .

[36]  B. V. K. Vijaya Kumar,et al.  Illumination Tolerant Face Recognition Using a Novel Face From Sketch Synthesis Approach and Advanced Correlation Filters , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[37]  Pong C. Yuen,et al.  Human Face Image Searching System Using Sketches , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[38]  Chunna Tian,et al.  Face Sketch Synthesis Algorithm Based on E-HMM and Selective Ensemble , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[39]  A. Martínez,et al.  The AR face databasae , 1998 .

[40]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Konstantinos N. Plataniotis,et al.  Regularization studies of linear discriminant analysis in small sample size scenarios with application to face recognition , 2005, Pattern Recognit. Lett..

[42]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Xiaogang Wang,et al.  Random Sampling for Subspace Face Recognition , 2006, International Journal of Computer Vision.