A Century of Portraits: A Visual Historical Record of American High School Yearbooks

Many details about our world are not captured in written records because they are too mundane or too abstract to describe in words. Fortunately, since the invention of the camera, an ever-increasing number of photographs capture much of this otherwise lost information. This plethora of artifacts documenting our "visual culture" is a treasure trove of knowledge as yet untapped by historians. We present a dataset of 37,921 frontal-facing American high school yearbook photos that allow us to use computation to glimpse into the historical visual record too voluminous to be evaluated manually. The collected portraits provide a constant visual frame of reference with varying content. We can therefore use them to consider issues such as a decade's defining style elements, or trends in fashion and social norms over time. We demonstrate that our historical image dataset may be used together with weakly-supervised data-driven techniques to perform scalable historical analysis of large image corpora with minimal human effort, much in the same way that large text corpora together with natural language processing revolutionized historians' workflow. Furthermore, we demonstrate the use of our dataset in dating grayscale portraits using deep learning methods.

[1]  Pierre-Yves Coulon,et al.  Frequential and color analysis for hair mask segmentation , 2008, 2008 15th IEEE International Conference on Image Processing.

[2]  Andrea Vedaldi,et al.  Understanding deep image representations by inverting them , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Pascal Vincent,et al.  Visualizing Higher-Layer Features of a Deep Network , 2009 .

[4]  Alexei A. Efros,et al.  What makes Paris look like Paris? , 2015, Commun. ACM.

[5]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[6]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[7]  Wen-Huang Cheng,et al.  What are the Fashion Trends in New York? , 2014, ACM Multimedia.

[8]  Bolei Zhou,et al.  Object Detectors Emerge in Deep Scene CNNs , 2014, ICLR.

[9]  C. Goldin,et al.  The Race between Education and Technology: The Evolution of U.S. Educational Wage Differentials, 1890 to 2005 , 2007 .

[10]  Alexei A. Efros,et al.  Dating Historical Color Images , 2012, ECCV.

[11]  C. Goldin America's Graduation from High School: The Evolution and Spread of Secondary Schooling in the Twentieth Century , 1998, The Journal of Economic History.

[12]  Fernando De la Torre,et al.  Learning Spatial and Temporal Cues for Multi-Label Facial Action Unit Detection , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[13]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[14]  Alexei A. Efros,et al.  Mid-level Visual Element Discovery as Discriminative Mode Seeking , 2013, NIPS.

[15]  Fernando De la Torre,et al.  Modeling Spatial and Temporal Cues for Multi-label Facial Action Unit Detection , 2016, ArXiv.

[16]  Connor Greenwell,et al.  Large-scale geo-facial image analysis , 2015, EURASIP J. Image Video Process..

[17]  Jitendra Malik,et al.  Discriminative Decorrelation for Clustering and Classification , 2012, ECCV.

[18]  Alexander Binder,et al.  Evaluating the Visualization of What a Deep Neural Network Has Learned , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Erez Lieberman Aiden,et al.  Quantitative Analysis of Culture Using Millions of Digitized Books , 2010, Science.

[20]  Alexei A. Efros,et al.  Linking Past to Present: Discovering Style in Two Centuries of Architecture , 2015, 2015 IEEE International Conference on Computational Photography (ICCP).

[21]  Victoria Sherrow,et al.  Encyclopedia of hair : a cultural history , 2006 .

[22]  Shaun J. Canavan,et al.  BP4D-Spontaneous: a high-resolution spontaneous 3D dynamic facial expression database , 2014, Image Vis. Comput..

[23]  E. Paluck,et al.  The contingent smile: a meta-analysis of sex differences in smiling. , 2003, Psychological bulletin.

[24]  Fernando De la Torre,et al.  Supervised Descent Method and Its Applications to Face Alignment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[26]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[27]  J. M. Ragan Gender displays in portrait photographs , 1982 .

[28]  Thomas Brox,et al.  Inverting Convolutional Networks with Convolutional Networks , 2015, ArXiv.

[29]  J. Girard Automatic Detection and Intensity Estimation of Spontaneous Smiles , 2014 .

[30]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[31]  Shree K. Nayar,et al.  Attribute and simile classifiers for face verification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[32]  Scott Workman,et al.  Analyzing human appearance as a cue for dating images , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[33]  Alexander C. Berg,et al.  Hipster Wars: Discovering Elements of Fashion Styles , 2014, ECCV.

[34]  Tinne Tuytelaars,et al.  Color features for dating historical color images , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[35]  Alexei A. Efros,et al.  A Century of Portraits: A Visual Historical Record of American High School Yearbooks , 2015, ICCV Workshops.

[36]  Yong Jae Lee,et al.  Style-Aware Mid-level Representation for Discovering Visual Connections in Space and Time , 2013, 2013 IEEE International Conference on Computer Vision.

[37]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[38]  Christina Kotchemidova Why We Say “Cheese”: Producing the Smile in Snapshot Photography , 2005 .

[39]  Hod Lipson,et al.  Understanding Neural Networks Through Deep Visualization , 2015, ArXiv.

[40]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .